[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:19 dist-test-slave-gjw7 krb5kdc[9021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:19 dist-test-slave-gjw7 krb5kdc[9021](info): set up 2 sockets
Aug 11 02:57:19 dist-test-slave-gjw7 krb5kdc[9021](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 02:57:20.255234 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:20 dist-test-slave-gjw7 krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:20.372155 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46855
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:20.482366 9046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:20.482615 9046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:20.482671 9046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:20.486569 9046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:20.486624 9046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:20.486650 9046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:20.486672 9046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:20.486696 9046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:20.491783 9046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:20.492777 9046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:20.493772 9046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:20.502442 9046 server_base.cc:1034] running on GCE node
W20260811 02:57:20.502403 9060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.502449 9064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.502408 9062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:20.503152 9046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:20.504199 9046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:20.505347 9046 hybrid_clock.cc:648] HybridClock initialized: now 1786417040505305 us; error 69 us; skew 500 ppm
Aug 11 02:57:20 dist-test-slave-gjw7 krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417040, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:20.508020 9046 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:20.508826 9046 webserver.cc:460] Webserver started at http://127.8.203.126:44563/ using document root <none> and password file <none>
I20260811 02:57:20.509392 9046 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:20.509464 9046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:20.509665 9046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:20.511679 9046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-gjw7"
server_key: "b3817c524e55855925b6edcaaf2f9beb"
server_key_iv: "2cfb2960f02a82c6bbf12a43ad99644b"
server_key_version: "encryptionkey@0"
I20260811 02:57:20.512224 9046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-gjw7"
server_key: "b3817c524e55855925b6edcaaf2f9beb"
server_key_iv: "2cfb2960f02a82c6bbf12a43ad99644b"
server_key_version: "encryptionkey@0"
I20260811 02:57:20.515775 9046 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:20.518102 9072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.519233 9046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:20.519354 9046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-gjw7"
server_key: "b3817c524e55855925b6edcaaf2f9beb"
server_key_iv: "2cfb2960f02a82c6bbf12a43ad99644b"
server_key_version: "encryptionkey@0"
I20260811 02:57:20.519448 9046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:20.531234 9046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:20.533784 9046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:20.533951 9046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:20.540664 9046 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:46855
I20260811 02:57:20.540704 9172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:46855 every 8 connection(s)
I20260811 02:57:20.541580 9046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:20.544201 9174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:20.547961 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 9046
I20260811 02:57:20.548123 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:20.549798 9005 external_mini_cluster.cc:1428] Setting key 99ab5678647faf730f9cc7e08505b1c1
I20260811 02:57:20.550251 9174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3: Bootstrap starting.
I20260811 02:57:20.553079 9174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:20.553920 9174 log.cc:824] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:20.555680 9174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3: No bootstrap required, opened a new log
I20260811 02:57:20.559650 9174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46855 } }
I20260811 02:57:20.559930 9174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:20.560006 9174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e883e7d2814e66a7b1fd4fcc2a35d3, State: Initialized, Role: FOLLOWER
I20260811 02:57:20.560469 9174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46855 } }
I20260811 02:57:20.560601 9174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:20.560667 9174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:20.560760 9174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:20.561635 9174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46855 } }
I20260811 02:57:20.561951 9174 leader_election.cc:302] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72e883e7d2814e66a7b1fd4fcc2a35d3; no voters:
I20260811 02:57:20.562222 9174 leader_election.cc:288] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:20.562361 9185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:20.562554 9185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [term 1 LEADER]: Becoming Leader. State: Replica: 72e883e7d2814e66a7b1fd4fcc2a35d3, State: Running, Role: LEADER
I20260811 02:57:20.562867 9185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46855 } }
I20260811 02:57:20.563457 9174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:20.564505 9188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72e883e7d2814e66a7b1fd4fcc2a35d3. Latest consensus state: current_term: 1 leader_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46855 } } }
I20260811 02:57:20.564505 9186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e883e7d2814e66a7b1fd4fcc2a35d3" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46855 } } }
I20260811 02:57:20.564664 9188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:20.564664 9186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3 [sys.catalog]: This master's current role is: LEADER
W20260811 02:57:20.568672 9212 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:20.568755 9212 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:20.568813 9201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 02:57:20 dist-test-slave-gjw7 krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:20.569706 9201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:20.575706 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:20.553269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.8.203.126:46855)
0811 02:57:20.553777 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:20.553797 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:20.554914 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 02:57:20.559205 (+ 4291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:20.559230 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:20.559621 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:20.559997 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:20.562295 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:20.563736 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:20.565529 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:20.566193 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:20.566352 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:20.570338 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:20.570343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:20.570356 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:20.570389 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:20.572114 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:20.572731 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:20.572737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:20.572743 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:20.572911 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:20.573309 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:20.573313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:20.573314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:20.573867 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:20.574051 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:20.574612 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 02:57:20.574871 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":132,"threads_started":1}
I20260811 02:57:20.578773 9201 catalog_manager.cc:1347] Generated new cluster ID: b2b81774435b430ba94eb511b7d614b0
I20260811 02:57:20.578832 9201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:20.590003 9201 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:20.590843 9201 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:20.604653 9201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72e883e7d2814e66a7b1fd4fcc2a35d3: Generated new TSK 0
I20260811 02:57:20.605504 9201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:20.633738 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46855
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:20.757241 9220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:20.757509 9220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:20.757556 9220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:20.761546 9220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:20.761606 9220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:20.761678 9220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:20.767194 9220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:20.768167 9220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:20.769174 9220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:20.778330 9231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.778344 9228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:20.778347 9233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:20.778460 9220 server_base.cc:1034] running on GCE node
I20260811 02:57:20.778924 9220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:20.779510 9220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:20.780684 9220 hybrid_clock.cc:648] HybridClock initialized: now 1786417040780669 us; error 31 us; skew 500 ppm
Aug 11 02:57:20 dist-test-slave-gjw7 krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417040, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:20.783632 9220 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:20.784406 9220 webserver.cc:460] Webserver started at http://127.8.203.65:39501/ using document root <none> and password file <none>
I20260811 02:57:20.784907 9220 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:20.784965 9220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:20.785149 9220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:20.787151 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "db6b1c4af2644c1bad8bd8cdcbed891f"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-gjw7"
server_key: "6fb0ccce30dea00d3ad38cab56b01a99"
server_key_iv: "8adf7d7e07e2ffe7b6668e4b710c52a1"
server_key_version: "encryptionkey@0"
I20260811 02:57:20.787685 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "db6b1c4af2644c1bad8bd8cdcbed891f"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-gjw7"
server_key: "6fb0ccce30dea00d3ad38cab56b01a99"
server_key_iv: "8adf7d7e07e2ffe7b6668e4b710c52a1"
server_key_version: "encryptionkey@0"
I20260811 02:57:20.791148 9220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 02:57:20.793386 9250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.794500 9220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:20.794602 9220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "db6b1c4af2644c1bad8bd8cdcbed891f"
format_stamp: "Formatted at 2026-08-11 02:57:20 on dist-test-slave-gjw7"
server_key: "6fb0ccce30dea00d3ad38cab56b01a99"
server_key_iv: "8adf7d7e07e2ffe7b6668e4b710c52a1"
server_key_version: "encryptionkey@0"
I20260811 02:57:20.794689 9220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:20.821273 9220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:20.824513 9220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:20.824692 9220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:20.825279 9220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:20.826428 9220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:20.826475 9220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.826527 9220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:20.826556 9220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:20.834976 9220 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:37489
I20260811 02:57:20.835013 9464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:37489 every 8 connection(s)
I20260811 02:57:20.835834 9220 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:20.840404 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 9220
I20260811 02:57:20.840492 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:20.840749 9005 external_mini_cluster.cc:1428] Setting key 459ae6e41af48a2710f9a6817c9a30b3
Aug 11 02:57:20 dist-test-slave-gjw7 krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417040, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:20.849400 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:20.837654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:35121 (local address 127.8.203.126:46855)
0811 02:57:20.837777 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:20.837781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:20.838573 (+ 792us) server_negotiation.cc:407] Connection header received
0811 02:57:20.839905 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:20.839908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:20.839970 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:20.840044 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:20.841249 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:20.841871 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:20.843066 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:20.843367 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:20.843468 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:20.845859 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:20.845863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:20.845866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:20.845893 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:20.847388 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:20.847857 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:20.847860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:20.847862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:20.847960 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:20.848242 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:20.848245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:20.848247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:20.848486 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:20.848591 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:20.849077 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 02:57:20.849202 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 02:57:20.850080 9467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:20.838001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46855 (local address 127.8.203.65:35121)
0811 02:57:20.838430 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:20.838472 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:20.839700 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:20.840223 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:20.840230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:20.840612 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:20.841106 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:20.841120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:20.842001 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:20.842005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:20.842944 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:20.842954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:20.843494 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:20.843496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:20.843638 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:20.844128 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:20.844144 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:20.845696 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:20.847494 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:20.847499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:20.847510 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:20.847758 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:20.848041 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:20.848043 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:20.848044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:20.848161 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:20.848582 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:20.848587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:20.848841 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 02:57:20.848993 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 02:57:20.849229 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:57:20.851207 9465 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46855
I20260811 02:57:20.851466 9465 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:20.851924 9465 heartbeater.cc:511] Master 127.8.203.126:46855 requested a full tablet report, sending...
I20260811 02:57:20.853345 9108 ts_manager.cc:194] Registered new tserver with Master: db6b1c4af2644c1bad8bd8cdcbed891f (127.8.203.65:37489)
I20260811 02:57:20.854501 9108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:35121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
W20260811 02:57:21.519238 9005 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.203.66: real 0.677s user 0.003s sys 0.001s
I20260811 02:57:21.519732 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46855
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:21.629135 9473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:21.629395 9473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:21.629453 9473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:21.633340 9473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:21.633388 9473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:21.633464 9473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:21.639824 9473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:21.640975 9473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:21.642021 9473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:21.652417 9482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:21.652402 9485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:21.652405 9487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:21.652876 9473 server_base.cc:1034] running on GCE node
I20260811 02:57:21.653241 9473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:21.653908 9473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:21.655055 9473 hybrid_clock.cc:648] HybridClock initialized: now 1786417041655028 us; error 37 us; skew 500 ppm
Aug 11 02:57:21 dist-test-slave-gjw7 krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417041, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:21.657754 9473 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:21.658568 9473 webserver.cc:460] Webserver started at http://127.8.203.66:33653/ using document root <none> and password file <none>
I20260811 02:57:21.659086 9473 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:21.659143 9473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:21.659333 9473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:21.661598 9473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "caf24498ae324a85ac4471783d8a9dec"
format_stamp: "Formatted at 2026-08-11 02:57:21 on dist-test-slave-gjw7"
server_key: "4d913e2e4204c6e9d12d73da3dee809b"
server_key_iv: "dded3321ff869a39cfd52fe8b4720324"
server_key_version: "encryptionkey@0"
I20260811 02:57:21.662132 9473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "caf24498ae324a85ac4471783d8a9dec"
format_stamp: "Formatted at 2026-08-11 02:57:21 on dist-test-slave-gjw7"
server_key: "4d913e2e4204c6e9d12d73da3dee809b"
server_key_iv: "dded3321ff869a39cfd52fe8b4720324"
server_key_version: "encryptionkey@0"
I20260811 02:57:21.666267 9473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:57:21.668504 9497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:21.669723 9473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:21.669823 9473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "caf24498ae324a85ac4471783d8a9dec"
format_stamp: "Formatted at 2026-08-11 02:57:21 on dist-test-slave-gjw7"
server_key: "4d913e2e4204c6e9d12d73da3dee809b"
server_key_iv: "dded3321ff869a39cfd52fe8b4720324"
server_key_version: "encryptionkey@0"
I20260811 02:57:21.669922 9473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:21.692173 9473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:21.694751 9473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:21.694923 9473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:21.695814 9473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:21.696642 9473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:21.696691 9473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:21.696741 9473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:21.696769 9473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:21.705506 9473 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:36443
I20260811 02:57:21.705534 9712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:36443 every 8 connection(s)
I20260811 02:57:21.706352 9473 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:57:21 dist-test-slave-gjw7 krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417041, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:21.716534 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 9473
I20260811 02:57:21.716642 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:21.716952 9005 external_mini_cluster.cc:1428] Setting key 67bb1404682eecc3fb0759f017c4aab1
I20260811 02:57:21.719864 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:21.708156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:52043 (local address 127.8.203.126:46855)
0811 02:57:21.708436 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:21.708440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:21.709111 (+ 671us) server_negotiation.cc:407] Connection header received
0811 02:57:21.710417 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:21.710420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:21.710472 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:21.710550 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:21.711629 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:21.712521 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:21.713663 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:21.713899 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:21.713974 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:21.716313 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:21.716316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:21.716318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:21.716340 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:21.717846 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:21.718384 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:21.718387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:21.718388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:21.718482 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:21.718799 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:21.718803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:21.718805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:21.719030 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:21.719125 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:21.719579 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 02:57:21.719702 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
I20260811 02:57:21.720590 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:21.708562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46855 (local address 127.8.203.66:52043)
0811 02:57:21.708977 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:21.709019 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:21.710227 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:21.710696 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:21.710703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:21.711042 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:21.711517 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:21.711530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:21.712623 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:21.712625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:21.713575 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:21.713585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:21.714011 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:21.714013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:21.714146 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:21.714674 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:21.714690 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:21.716155 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:21.717953 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:21.717959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:21.717973 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:21.718295 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:21.718574 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:21.718577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:21.718578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:21.718710 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:21.719116 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:21.719120 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:21.719334 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 02:57:21.719517 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 02:57:21.719755 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":104,"threads_started":1}
I20260811 02:57:21.721657 9713 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46855
I20260811 02:57:21.721882 9713 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:21.722347 9713 heartbeater.cc:511] Master 127.8.203.126:46855 requested a full tablet report, sending...
I20260811 02:57:21.723357 9108 ts_manager.cc:194] Registered new tserver with Master: caf24498ae324a85ac4471783d8a9dec (127.8.203.66:36443)
I20260811 02:57:21.723906 9108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:52043
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
I20260811 02:57:21.856680 9465 heartbeater.cc:503] Master 127.8.203.126:46855 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
W20260811 02:57:22.517887 9005 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.203.67: real 0.799s user 0.003s sys 0.001s
I20260811 02:57:22.518432 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46855
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:22.628624 9724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:22.628854 9724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:22.628892 9724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:22.632788 9724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:22.632836 9724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:22.632905 9724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:22.637923 9724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:22.638825 9724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:22.639762 9724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:22.648850 9737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:22.648891 9739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:22.648981 9724 server_base.cc:1034] running on GCE node
W20260811 02:57:22.649219 9736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:22.649637 9724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:22.650290 9724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:22.651441 9724 hybrid_clock.cc:648] HybridClock initialized: now 1786417042651412 us; error 36 us; skew 500 ppm
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:22.656867 9724 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:22.657769 9724 webserver.cc:460] Webserver started at http://127.8.203.67:38987/ using document root <none> and password file <none>
I20260811 02:57:22.658295 9724 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:22.658360 9724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:22.658536 9724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:22.660588 9724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "137971b6279c400780df7d436b99d6ff"
format_stamp: "Formatted at 2026-08-11 02:57:22 on dist-test-slave-gjw7"
server_key: "4281ac029a275a84692687b44e701543"
server_key_iv: "cd2176976fde482fe280174fe5690db4"
server_key_version: "encryptionkey@0"
I20260811 02:57:22.661139 9724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "137971b6279c400780df7d436b99d6ff"
format_stamp: "Formatted at 2026-08-11 02:57:22 on dist-test-slave-gjw7"
server_key: "4281ac029a275a84692687b44e701543"
server_key_iv: "cd2176976fde482fe280174fe5690db4"
server_key_version: "encryptionkey@0"
I20260811 02:57:22.664985 9724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:57:22.667462 9751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:22.668629 9724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:22.668761 9724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "137971b6279c400780df7d436b99d6ff"
format_stamp: "Formatted at 2026-08-11 02:57:22 on dist-test-slave-gjw7"
server_key: "4281ac029a275a84692687b44e701543"
server_key_iv: "cd2176976fde482fe280174fe5690db4"
server_key_version: "encryptionkey@0"
I20260811 02:57:22.668920 9724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:22.699910 9724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:22.702915 9724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:22.703078 9724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:22.703635 9724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:22.704486 9724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:22.704541 9724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:22.704610 9724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:22.704633 9724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:22.714022 9724 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43693
I20260811 02:57:22.714064 9995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43693 every 8 connection(s)
I20260811 02:57:22.714937 9724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:22.715276 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 9724
I20260811 02:57:22.715377 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:22.715672 9005 external_mini_cluster.cc:1428] Setting key 68ab8628b00d70ae430cad9e645a3f69
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:22.725795 9713 heartbeater.cc:503] Master 127.8.203.126:46855 was elected leader, sending a full tablet report...
I20260811 02:57:22.728377 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:22.716967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56351 (local address 127.8.203.126:46855)
0811 02:57:22.717255 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:22.717260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:22.719553 (+ 2293us) server_negotiation.cc:407] Connection header received
0811 02:57:22.719590 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:22.719592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:22.719639 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:22.719704 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:22.720897 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.721475 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.722582 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.722815 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.722901 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:22.725136 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:22.725140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:22.725142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:22.725166 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:22.726665 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:22.727173 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:22.727176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:22.727177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:22.727259 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:22.727516 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:22.727518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:22.727519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:22.727682 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:22.727750 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:22.728159 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 02:57:22.728253 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":99,"threads_started":1}
I20260811 02:57:22.729223 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:22.717512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46855 (local address 127.8.203.67:56351)
0811 02:57:22.718015 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:22.718057 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:22.719330 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:22.719900 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:22.719915 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:22.720281 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:22.720770 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:22.720784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.721586 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.721589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:22.722477 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:22.722487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.722936 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.722938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:22.723073 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:22.723607 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:22.723623 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:22.724989 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:22.726790 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:22.726796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:22.726810 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:22.727088 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:22.727327 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:22.727329 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:22.727331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:22.727452 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:22.727750 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:22.727754 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:22.727981 (+ 227us) client_negotiation.cc:769] Sending connection context
0811 02:57:22.728129 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 02:57:22.728364 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":146,"threads_started":1}
I20260811 02:57:22.730309 9996 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46855
I20260811 02:57:22.730522 9996 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:22.730978 9996 heartbeater.cc:511] Master 127.8.203.126:46855 requested a full tablet report, sending...
I20260811 02:57:22.731869 9106 ts_manager.cc:194] Registered new tserver with Master: 137971b6279c400780df7d436b99d6ff (127.8.203.67:43693)
I20260811 02:57:22.732386 9106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:56351
I20260811 02:57:22.741664 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:22.767151 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:22.758590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44968 (local address 127.8.203.65:37489)
0811 02:57:22.758866 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:22.758884 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:22.758907 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:57:22.758957 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:22.758962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:22.759145 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:22.759256 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:22.759835 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.760565 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.761336 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.761695 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.761772 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:22.763962 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:22.763972 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:22.763986 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:22.764011 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:22.765630 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:22.766033 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:22.766038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:22.766040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:22.766118 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:22.766301 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:22.766303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:22.766304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:22.766529 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:22.766683 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:22.766815 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 02:57:22.766957 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":84,"threads_started":1}
W20260811 02:57:22.768229 9305 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44968
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[9021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[9021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:22.793197 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:22.785683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.8.203.65:37489)
0811 02:57:22.785831 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:22.785844 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:22.785929 (+ 85us) server_negotiation.cc:407] Connection header received
0811 02:57:22.786066 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:22.786069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:22.786114 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:22.786178 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:22.786852 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.787356 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.788150 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:22.788379 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:22.788464 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:22.790630 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:22.790634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:22.790636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:22.790660 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:22.791716 (+ 1056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:22.792084 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:22.792088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:22.792090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:22.792166 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:22.792424 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:22.792427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:22.792429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:22.792595 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:22.792712 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:22.792929 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 02:57:22.793043 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:57:22.794200 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 9220
I20260811 02:57:22.818454 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 9473
I20260811 02:57:22.856791 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 9724
I20260811 02:57:22.867188 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 9046
2026-08-11T02:57:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3848 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[10028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[10028](info): set up 2 sockets
Aug 11 02:57:22 dist-test-slave-gjw7 krb5kdc[10028](info): commencing operation
krb5kdc: starting...
W20260811 02:57:24.083468 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:24 dist-test-slave-gjw7 krb5kdc[10028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:24.213514 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43851
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:39037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:24.321264 10055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:24.321528 10055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:24.321579 10055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:24.325414 10055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:24.325459 10055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:24.325484 10055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:24.325503 10055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:24.325524 10055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:24.330513 10055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:39037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:24.331454 10055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:24.332382 10055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:24.340438 10061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.340431 10065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.340431 10062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:24.340683 10055 server_base.cc:1034] running on GCE node
I20260811 02:57:24.341213 10055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:24.342270 10055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:24.343429 10055 hybrid_clock.cc:648] HybridClock initialized: now 1786417044343408 us; error 35 us; skew 500 ppm
Aug 11 02:57:24 dist-test-slave-gjw7 krb5kdc[10028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:24.345952 10055 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:24.346772 10055 webserver.cc:460] Webserver started at http://127.8.203.126:35645/ using document root <none> and password file <none>
I20260811 02:57:24.347258 10055 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:24.347312 10055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:24.347487 10055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:24.349413 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "80d05f6a3a6b42a1ba479c96a03de05c"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "16a98bbedf4770b0d48b3aa9917502eb"
server_key_iv: "076eb3ed756b2cdc08d3d7ffd82607f9"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.349937 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "80d05f6a3a6b42a1ba479c96a03de05c"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "16a98bbedf4770b0d48b3aa9917502eb"
server_key_iv: "076eb3ed756b2cdc08d3d7ffd82607f9"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.353448 10055 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:24.355679 10080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.356849 10055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:24.356952 10055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "80d05f6a3a6b42a1ba479c96a03de05c"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "16a98bbedf4770b0d48b3aa9917502eb"
server_key_iv: "076eb3ed756b2cdc08d3d7ffd82607f9"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.357049 10055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:24.367698 10055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:24.370194 10055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:24.370344 10055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:24.376884 10055 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:43851
I20260811 02:57:24.376916 10195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:43851 every 8 connection(s)
I20260811 02:57:24.377826 10055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:24.378387 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10055
I20260811 02:57:24.378499 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:24.378726 9005 external_mini_cluster.cc:1428] Setting key 3c83a194f56d5a9afea11083bb5f28c1
I20260811 02:57:24.380826 10198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:24.386176 10198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c: Bootstrap starting.
Aug 11 02:57:24 dist-test-slave-gjw7 krb5kdc[10028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:24.388407 10198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:24.389094 10198 log.cc:824] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:24.391059 10198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c: No bootstrap required, opened a new log
I20260811 02:57:24.391503 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:24.380024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.8.203.126:43851)
0811 02:57:24.380466 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:24.380481 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:24.380514 (+ 33us) server_negotiation.cc:407] Connection header received
0811 02:57:24.381147 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:24.381165 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:24.381519 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:24.381814 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:24.382569 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.383553 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.384377 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.384823 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.384966 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:24.387279 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:24.387284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:24.387296 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:24.387322 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:24.388857 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:24.389246 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:24.389251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:24.389255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:24.389405 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:24.389694 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:24.389698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:24.389700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:24.390094 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:24.390240 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:24.390535 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 02:57:24.390790 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":156,"threads_started":1}
I20260811 02:57:24.394459 10198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43851 } }
I20260811 02:57:24.394668 10198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:24.394734 10198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d05f6a3a6b42a1ba479c96a03de05c, State: Initialized, Role: FOLLOWER
I20260811 02:57:24.395165 10198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43851 } }
I20260811 02:57:24.395325 10198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:24.395399 10198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:24.395496 10198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:24.396332 10198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43851 } }
I20260811 02:57:24.396642 10198 leader_election.cc:302] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80d05f6a3a6b42a1ba479c96a03de05c; no voters:
I20260811 02:57:24.396915 10198 leader_election.cc:288] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:24.397007 10206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:24.397212 10206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [term 1 LEADER]: Becoming Leader. State: Replica: 80d05f6a3a6b42a1ba479c96a03de05c, State: Running, Role: LEADER
I20260811 02:57:24.397526 10206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43851 } }
I20260811 02:57:24.398036 10198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:24.399006 10207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43851 } } }
I20260811 02:57:24.399134 10207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:24.399443 10208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80d05f6a3a6b42a1ba479c96a03de05c. Latest consensus state: current_term: 1 leader_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d05f6a3a6b42a1ba479c96a03de05c" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43851 } } }
I20260811 02:57:24.399535 10208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:24.399590 10225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:24.402662 10225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:24.408004 10225 catalog_manager.cc:1347] Generated new cluster ID: 351211f799ca42d383028e59db7e8525
I20260811 02:57:24.408071 10225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:24.422016 10225 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:24.422850 10225 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:24.429416 10225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80d05f6a3a6b42a1ba479c96a03de05c: Generated new TSK 0
I20260811 02:57:24.430074 10225 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:24.466434 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43851
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:39037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:24.576066 10249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:24.576305 10249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:24.576351 10249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:24.580216 10249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:24.580262 10249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:24.580317 10249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:24.585327 10249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:39037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43851
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:24.586236 10249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:24.587159 10249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:24.596308 10257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.596315 10264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:24.596364 10249 server_base.cc:1034] running on GCE node
W20260811 02:57:24.596356 10258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:24.596917 10249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:24.597539 10249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:24.598685 10249 hybrid_clock.cc:648] HybridClock initialized: now 1786417044598660 us; error 37 us; skew 500 ppm
Aug 11 02:57:24 dist-test-slave-gjw7 krb5kdc[10028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:24.601226 10249 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:24.602027 10249 webserver.cc:460] Webserver started at http://127.8.203.65:38591/ using document root <none> and password file <none>
I20260811 02:57:24.602546 10249 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:24.602609 10249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:24.602810 10249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:24.604759 10249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "4dca84a3085d4526b0e960a08160023b"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "5a21a32577fb51c1149e6a18e36b1c7d"
server_key_iv: "9a57bcea610149c4560a6a5380de2a3a"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.605314 10249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "4dca84a3085d4526b0e960a08160023b"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "5a21a32577fb51c1149e6a18e36b1c7d"
server_key_iv: "9a57bcea610149c4560a6a5380de2a3a"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.608675 10249 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:24.610945 10276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.612071 10249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:24.612172 10249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "4dca84a3085d4526b0e960a08160023b"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "5a21a32577fb51c1149e6a18e36b1c7d"
server_key_iv: "9a57bcea610149c4560a6a5380de2a3a"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.612260 10249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:24.637233 10249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:24.640208 10249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:24.640355 10249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:24.640877 10249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:24.641702 10249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:24.641757 10249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.641800 10249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:24.641821 10249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.649956 10249 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:33147
I20260811 02:57:24.650015 10521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:33147 every 8 connection(s)
I20260811 02:57:24.650805 10249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:24.652402 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10249
I20260811 02:57:24.652501 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:24.652761 9005 external_mini_cluster.cc:1428] Setting key 700b890f5dd17beb3eb44032c9413657
Aug 11 02:57:24 dist-test-slave-gjw7 krb5kdc[10028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:57:24.664320 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:24.652560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:52553 (local address 127.8.203.126:43851)
0811 02:57:24.652716 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:24.652721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:24.653639 (+ 918us) server_negotiation.cc:407] Connection header received
0811 02:57:24.654634 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:24.654637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:24.654680 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:24.654744 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:24.656174 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.656955 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.658101 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.658343 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.658433 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:24.661036 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:24.661039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:24.661042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:24.661065 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:24.662361 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:24.662919 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:24.662921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:24.662922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:24.662991 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:24.663311 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:24.663313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:24.663314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:24.663531 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:24.663613 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:24.664084 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 02:57:24.664196 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:57:24.665107 10527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:24.652973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43851 (local address 127.8.203.65:52553)
0811 02:57:24.653496 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:24.653551 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:24.654414 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:24.654939 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:24.654947 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:24.655391 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:24.656038 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:24.656058 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.657094 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.657097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:24.657979 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:24.657989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.658482 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.658486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:24.658646 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:24.659202 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:24.659218 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:24.660872 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:24.662476 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:24.662483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:24.662498 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:24.662831 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:24.663066 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:24.663069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:24.663071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:24.663236 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:24.663647 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:24.663653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:24.663895 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 02:57:24.664049 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 02:57:24.664332 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 02:57:24.666337 10523 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43851
I20260811 02:57:24.666591 10523 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:24.667161 10523 heartbeater.cc:511] Master 127.8.203.126:43851 requested a full tablet report, sending...
I20260811 02:57:24.668852 10116 ts_manager.cc:194] Registered new tserver with Master: 4dca84a3085d4526b0e960a08160023b (127.8.203.65:33147)
I20260811 02:57:24.670058 10116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:52553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:24.679616 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43851
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:39037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:24.787648 10532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:24.787874 10532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:24.787916 10532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:24.791779 10532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:24.791829 10532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:24.791915 10532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:24.796921 10532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:39037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43851
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:24.797891 10532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:24.798863 10532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:24.807888 10541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.807905 10539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:24.807902 10547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:24.808097 10532 server_base.cc:1034] running on GCE node
I20260811 02:57:24.808507 10532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:24.809041 10532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:24.810184 10532 hybrid_clock.cc:648] HybridClock initialized: now 1786417044810150 us; error 48 us; skew 500 ppm
Aug 11 02:57:24 dist-test-slave-gjw7 krb5kdc[10028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:24.812676 10532 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:24.813501 10532 webserver.cc:460] Webserver started at http://127.8.203.66:37045/ using document root <none> and password file <none>
I20260811 02:57:24.814009 10532 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:24.814072 10532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:24.814273 10532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:24.816277 10532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "289bb4b8c9414910bfd26f3b757337fb"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "6bb5b8d440e1ef51e6bc7f196b4045d1"
server_key_iv: "2806ed6cd11e0505cb696cd97bf857fb"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.816787 10532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "289bb4b8c9414910bfd26f3b757337fb"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "6bb5b8d440e1ef51e6bc7f196b4045d1"
server_key_iv: "2806ed6cd11e0505cb696cd97bf857fb"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.820188 10532 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:24.822472 10559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.823560 10532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:24.823662 10532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "289bb4b8c9414910bfd26f3b757337fb"
format_stamp: "Formatted at 2026-08-11 02:57:24 on dist-test-slave-gjw7"
server_key: "6bb5b8d440e1ef51e6bc7f196b4045d1"
server_key_iv: "2806ed6cd11e0505cb696cd97bf857fb"
server_key_version: "encryptionkey@0"
I20260811 02:57:24.823747 10532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:24.839718 10532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:24.842554 10532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:24.842703 10532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:24.843209 10532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:24.843933 10532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:24.843977 10532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.844026 10532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:24.844053 10532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:24.851810 10532 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:43897
I20260811 02:57:24.851843 10813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:43897 every 8 connection(s)
I20260811 02:57:24.852593 10532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:24.855343 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10532
I20260811 02:57:24.855459 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:24.855705 9005 external_mini_cluster.cc:1428] Setting key 419f92fe6acbc57bcc965533416a6ffb
Aug 11 02:57:24 dist-test-slave-gjw7 krb5kdc[10028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 02:57:24.865379 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:24.854279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:60567 (local address 127.8.203.126:43851)
0811 02:57:24.854444 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:24.854449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:24.855166 (+ 717us) server_negotiation.cc:407] Connection header received
0811 02:57:24.856508 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:24.856511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:24.856554 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:24.856621 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:24.857860 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.858358 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.859453 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.859669 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.859753 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:24.862187 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:24.862190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:24.862192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:24.862215 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:24.863430 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:24.863976 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:24.863979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:24.863982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:24.864058 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:24.864361 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:24.864364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:24.864366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:24.864533 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:24.864616 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:24.865073 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 02:57:24.865193 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:57:24.866137 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:24.854628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43851 (local address 127.8.203.66:60567)
0811 02:57:24.855034 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:24.855073 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:24.856308 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:24.856807 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:24.856816 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:24.857242 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:24.857726 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:24.857739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.858479 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.858483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:24.859325 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:24.859335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:24.859799 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:24.859803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:24.859966 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:24.860470 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:24.860490 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:24.862038 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:24.863534 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:24.863540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:24.863555 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:24.863854 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:24.864140 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:24.864144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:24.864146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:24.864275 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:24.864630 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:24.864636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:24.864852 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 02:57:24.865011 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 02:57:24.865329 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":99,"threads_started":1}
I20260811 02:57:24.867141 10815 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43851
I20260811 02:57:24.867360 10815 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
I20260811 02:57:24.867872 10815 heartbeater.cc:511] Master 127.8.203.126:43851 requested a full tablet report, sending...
I20260811 02:57:24.868865 10116 ts_manager.cc:194] Registered new tserver with Master: 289bb4b8c9414910bfd26f3b757337fb (127.8.203.66:43897)
I20260811 02:57:24.869504 10116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:60567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:24.880857 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43851
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:39037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:24.991315 10827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:24.991555 10827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:24.991595 10827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:24.995522 10827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:24.995579 10827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:24.995635 10827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:25.000869 10827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:39037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43851
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:25.001863 10827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:25.002811 10827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:25.011966 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:25.011978 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:25.012104 10843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:25.012181 10827 server_base.cc:1034] running on GCE node
I20260811 02:57:25.012631 10827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:25.013188 10827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:25.014341 10827 hybrid_clock.cc:648] HybridClock initialized: now 1786417045014326 us; error 31 us; skew 500 ppm
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[10028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417045, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:25.017032 10827 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:25.017885 10827 webserver.cc:460] Webserver started at http://127.8.203.67:44217/ using document root <none> and password file <none>
I20260811 02:57:25.018406 10827 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:25.018464 10827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:25.018630 10827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:25.020542 10827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "80db8971e5bc4e17a255cb9036b42a47"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-gjw7"
server_key: "ad05967c06ecf18bfe51c9d3289e6fdd"
server_key_iv: "ee5a8b744563053ace3183385aa9ff15"
server_key_version: "encryptionkey@0"
I20260811 02:57:25.021056 10827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "80db8971e5bc4e17a255cb9036b42a47"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-gjw7"
server_key: "ad05967c06ecf18bfe51c9d3289e6fdd"
server_key_iv: "ee5a8b744563053ace3183385aa9ff15"
server_key_version: "encryptionkey@0"
I20260811 02:57:25.024567 10827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:25.026898 10856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:25.028049 10827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:25.028143 10827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "80db8971e5bc4e17a255cb9036b42a47"
format_stamp: "Formatted at 2026-08-11 02:57:25 on dist-test-slave-gjw7"
server_key: "ad05967c06ecf18bfe51c9d3289e6fdd"
server_key_iv: "ee5a8b744563053ace3183385aa9ff15"
server_key_version: "encryptionkey@0"
I20260811 02:57:25.028244 10827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:25.041038 10827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:25.043557 10827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:25.043699 10827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:25.044215 10827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:25.044970 10827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:25.045014 10827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:25.045053 10827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:25.045071 10827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:25.053362 10827 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:42011
I20260811 02:57:25.053396 11073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:42011 every 8 connection(s)
I20260811 02:57:25.054204 10827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:25.056964 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10827
I20260811 02:57:25.057080 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:25.057415 9005 external_mini_cluster.cc:1428] Setting key 872fbc562cc6dba1d47be3f902b445f7
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[10028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417045, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:25.066802 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.055911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:45033 (local address 127.8.203.126:43851)
0811 02:57:25.056048 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.056052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.056929 (+ 877us) server_negotiation.cc:407] Connection header received
0811 02:57:25.058063 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.058066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.058123 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.058199 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:25.059582 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.060123 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.061263 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.061544 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.061635 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.063811 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:25.063814 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:25.063816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:25.063840 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:25.065186 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.065673 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.065675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.065677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.065748 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.065994 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.065996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.065997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.066151 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:25.066218 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.066599 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.066689 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:57:25.067582 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.056274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43851 (local address 127.8.203.67:45033)
0811 02:57:25.056744 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:25.056802 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:25.057827 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:25.058414 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:25.058424 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:25.058896 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:25.059439 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.059461 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.060235 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.060238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.061166 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.061178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.061682 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.061693 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.061831 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.062323 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:25.062338 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:25.063656 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:25.065320 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:25.065328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:25.065342 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:25.065582 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:25.065813 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:25.065815 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:25.065817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:25.065934 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:25.066219 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:25.066223 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:25.066427 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 02:57:25.066568 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 02:57:25.066802 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260811 02:57:25.068614 11079 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43851
I20260811 02:57:25.068831 11079 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:25.069276 11079 heartbeater.cc:511] Master 127.8.203.126:43851 requested a full tablet report, sending...
I20260811 02:57:25.070348 10116 ts_manager.cc:194] Registered new tserver with Master: 80db8971e5bc4e17a255cb9036b42a47 (127.8.203.67:42011)
I20260811 02:57:25.070948 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:57:25.071015 10116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:45033
I20260811 02:57:25.079967 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.074216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.8.203.126:43851)
0811 02:57:25.074317 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.074320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.074451 (+ 131us) server_negotiation.cc:407] Connection header received
0811 02:57:25.074533 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.074534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.074567 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.074626 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:25.075161 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.075624 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.076323 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.076517 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.076579 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.077651 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:25.077654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:25.077656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:25.077675 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:25.078851 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.079149 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.079151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.079152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.079213 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.079374 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.079376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.079376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.079550 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:25.079642 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.079777 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.079864 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 02:57:25.083912 10116 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:25.086766 10116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:25.100240 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.094949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39116 (local address 127.8.203.67:42011)
0811 02:57:25.095392 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.095403 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.095425 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:57:25.095496 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.095500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.095691 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.096153 (+ 462us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:25.096271 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.097476 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.098929 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.099594 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.099688 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.099825 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.099997 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.100088 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":96,"threads_started":1}
I20260811 02:57:25.100265 11099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.094574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42011 (local address 127.0.0.1:39116)
0811 02:57:25.094969 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:25.095024 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:25.095158 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:25.095854 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:25.095859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:25.095881 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:25.096022 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.096029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.097607 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.097611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.098819 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.098827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.099702 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.099705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.099808 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.099859 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 02:57:25.099959 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 02:57:25.100111 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":54,"spinlock_wait_cycles":6144,"thread_start_us":92,"threads_started":1}
I20260811 02:57:25.100402 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.094345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:33147 (local address 127.0.0.1:56032)
0811 02:57:25.095315 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:25.095326 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:25.095396 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:25.096057 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:25.096060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:25.096078 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:25.096206 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.096213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.097639 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.097641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.098818 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.098828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.099944 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.099947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.100089 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.100117 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:57:25.100186 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 02:57:25.100265 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":131,"thread_start_us":106,"threads_started":1}
I20260811 02:57:25.100499 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.094534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.8.203.66:43897)
0811 02:57:25.095473 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.095484 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.095505 (+ 21us) server_negotiation.cc:407] Connection header received
0811 02:57:25.095579 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.095584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.095771 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.095946 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:25.096351 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.097109 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.098757 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.099634 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.099738 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.099872 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.100271 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.100356 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":126,"threads_started":1}
I20260811 02:57:25.100775 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.094234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43897 (local address 127.0.0.1:39474)
0811 02:57:25.094969 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:25.095022 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:25.095160 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:25.096099 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:25.096102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:25.096115 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:25.096217 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.096222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.097223 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.097226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.098618 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.098629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.099938 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.099941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.100123 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.100142 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:57:25.100197 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 02:57:25.100660 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":450,"spinlock_wait_cycles":1792,"thread_start_us":65,"threads_started":1}
I20260811 02:57:25.100898 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.094914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.8.203.65:33147)
0811 02:57:25.095628 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.095638 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.095657 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:57:25.095723 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.095728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.095905 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.096066 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:25.096323 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.097467 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.098958 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.099689 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.099770 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.099895 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.100656 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.100741 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":117,"threads_started":1}
I20260811 02:57:25.103472 10672 tablet_service.cc:1467] Processing CreateTablet for tablet 7dfe037a507b42ec899e359c3559789d (DEFAULT_TABLE table=test-table [id=ff67777351ed4379bd7b4bf1aba893d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:25.103561 10934 tablet_service.cc:1467] Processing CreateTablet for tablet 7dfe037a507b42ec899e359c3559789d (DEFAULT_TABLE table=test-table [id=ff67777351ed4379bd7b4bf1aba893d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:25.103561 10377 tablet_service.cc:1467] Processing CreateTablet for tablet 7dfe037a507b42ec899e359c3559789d (DEFAULT_TABLE table=test-table [id=ff67777351ed4379bd7b4bf1aba893d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:25.104394 10672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dfe037a507b42ec899e359c3559789d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:25.104559 10377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dfe037a507b42ec899e359c3559789d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:25.104559 10934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dfe037a507b42ec899e359c3559789d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:25.111186 11106 tablet_bootstrap.cc:492] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b: Bootstrap starting.
I20260811 02:57:25.111429 11105 tablet_bootstrap.cc:492] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb: Bootstrap starting.
I20260811 02:57:25.112056 11107 tablet_bootstrap.cc:492] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47: Bootstrap starting.
I20260811 02:57:25.113561 11106 tablet_bootstrap.cc:654] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:25.113720 11107 tablet_bootstrap.cc:654] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:25.113989 11105 tablet_bootstrap.cc:654] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:25.114711 11107 log.cc:824] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:25.114756 11106 log.cc:824] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:25.114756 11105 log.cc:824] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:25.116598 11107 tablet_bootstrap.cc:492] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47: No bootstrap required, opened a new log
I20260811 02:57:25.116778 11107 ts_tablet_manager.cc:1397] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:25.116977 11106 tablet_bootstrap.cc:492] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b: No bootstrap required, opened a new log
I20260811 02:57:25.117116 11105 tablet_bootstrap.cc:492] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb: No bootstrap required, opened a new log
I20260811 02:57:25.117167 11106 ts_tablet_manager.cc:1397] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:25.117276 11105 ts_tablet_manager.cc:1397] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 02:57:25.119562 11107 raft_consensus.cc:348] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.119758 11107 raft_consensus.cc:374] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:25.119788 11107 raft_consensus.cc:729] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80db8971e5bc4e17a255cb9036b42a47, State: Initialized, Role: FOLLOWER
I20260811 02:57:25.120147 11107 consensus_queue.cc:260] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.120698 11079 heartbeater.cc:503] Master 127.8.203.126:43851 was elected leader, sending a full tablet report...
I20260811 02:57:25.121042 11107 ts_tablet_manager.cc:1428] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:25.120890 11106 raft_consensus.cc:348] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.120890 11105 raft_consensus.cc:348] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.121127 11105 raft_consensus.cc:374] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:25.121127 11106 raft_consensus.cc:374] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:25.121176 11106 raft_consensus.cc:729] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dca84a3085d4526b0e960a08160023b, State: Initialized, Role: FOLLOWER
I20260811 02:57:25.121176 11105 raft_consensus.cc:729] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 289bb4b8c9414910bfd26f3b757337fb, State: Initialized, Role: FOLLOWER
I20260811 02:57:25.121618 11105 consensus_queue.cc:260] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.121618 11106 consensus_queue.cc:260] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.122449 10523 heartbeater.cc:503] Master 127.8.203.126:43851 was elected leader, sending a full tablet report...
I20260811 02:57:25.122555 11105 ts_tablet_manager.cc:1428] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260811 02:57:25.123322 11106 ts_tablet_manager.cc:1428] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:25.123562 10815 heartbeater.cc:503] Master 127.8.203.126:43851 was elected leader, sending a full tablet report...
W20260811 02:57:25.151811 10526 tablet.cc:2369] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:25.305115 11081 tablet.cc:2369] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:25.353600 10817 tablet.cc:2369] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:25.468814 11119 raft_consensus.cc:482] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:25.469048 11119 raft_consensus.cc:504] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.470227 11119 leader_election.cc:288] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 289bb4b8c9414910bfd26f3b757337fb (127.8.203.66:43897), 80db8971e5bc4e17a255cb9036b42a47 (127.8.203.67:42011)
I20260811 02:57:25.474018 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.470651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:52075 (local address 127.8.203.66:43897)
0811 02:57:25.470800 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.470805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.470911 (+ 106us) server_negotiation.cc:407] Connection header received
0811 02:57:25.471043 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.471045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.471090 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.471149 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:25.471422 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.471950 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.473048 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.473476 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.473572 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.473631 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.473811 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.473887 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:57:25.474025 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.470699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:37831 (local address 127.8.203.67:42011)
0811 02:57:25.470831 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.470836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.471027 (+ 191us) server_negotiation.cc:407] Connection header received
0811 02:57:25.471171 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.471174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.471214 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.471272 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:25.471523 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.472081 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.473076 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.473505 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.473615 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.473670 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.473853 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.473921 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:57:25.474040 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.470451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43897 (local address 127.8.203.65:52075)
0811 02:57:25.470812 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:25.470826 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:25.470941 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:25.471176 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:25.471179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:25.471195 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:25.471314 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.471319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.472058 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.472061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.472922 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.472932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.473586 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.473590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.473687 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.473716 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:57:25.473779 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 02:57:25.473852 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":97,"threads_started":1}
I20260811 02:57:25.474045 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.470451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42011 (local address 127.8.203.65:37831)
0811 02:57:25.470933 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:25.470946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:25.471062 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:25.471291 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:25.471294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:25.471309 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:25.471428 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.471434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.472193 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.472196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.472956 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:25.472965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.473633 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.473637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:25.473734 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.473752 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:57:25.473827 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:57:25.473891 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":45,"thread_start_us":112,"threads_started":1}
I20260811 02:57:25.474727 10712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfe037a507b42ec899e359c3559789d" candidate_uuid: "4dca84a3085d4526b0e960a08160023b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "289bb4b8c9414910bfd26f3b757337fb" is_pre_election: true
I20260811 02:57:25.474845 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfe037a507b42ec899e359c3559789d" candidate_uuid: "4dca84a3085d4526b0e960a08160023b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80db8971e5bc4e17a255cb9036b42a47" is_pre_election: true
I20260811 02:57:25.475047 10712 raft_consensus.cc:2454] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dca84a3085d4526b0e960a08160023b in term 0.
I20260811 02:57:25.475070 10982 raft_consensus.cc:2454] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dca84a3085d4526b0e960a08160023b in term 0.
I20260811 02:57:25.475551 10282 leader_election.cc:302] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 289bb4b8c9414910bfd26f3b757337fb, 4dca84a3085d4526b0e960a08160023b, 80db8971e5bc4e17a255cb9036b42a47; no voters:
I20260811 02:57:25.475811 11119 raft_consensus.cc:2781] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:25.475898 11119 raft_consensus.cc:482] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:25.475960 11119 raft_consensus.cc:3037] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:25.476955 11119 raft_consensus.cc:504] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.477234 11119 leader_election.cc:288] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [CANDIDATE]: Term 1 election: Requested vote from peers 289bb4b8c9414910bfd26f3b757337fb (127.8.203.66:43897), 80db8971e5bc4e17a255cb9036b42a47 (127.8.203.67:42011)
I20260811 02:57:25.477591 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfe037a507b42ec899e359c3559789d" candidate_uuid: "4dca84a3085d4526b0e960a08160023b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80db8971e5bc4e17a255cb9036b42a47"
I20260811 02:57:25.477613 10712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfe037a507b42ec899e359c3559789d" candidate_uuid: "4dca84a3085d4526b0e960a08160023b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "289bb4b8c9414910bfd26f3b757337fb"
I20260811 02:57:25.477715 10712 raft_consensus.cc:3037] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:25.477716 10982 raft_consensus.cc:3037] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:25.478917 10712 raft_consensus.cc:2454] T 7dfe037a507b42ec899e359c3559789d P 289bb4b8c9414910bfd26f3b757337fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dca84a3085d4526b0e960a08160023b in term 1.
I20260811 02:57:25.478917 10982 raft_consensus.cc:2454] T 7dfe037a507b42ec899e359c3559789d P 80db8971e5bc4e17a255cb9036b42a47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dca84a3085d4526b0e960a08160023b in term 1.
I20260811 02:57:25.479198 10282 leader_election.cc:302] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dca84a3085d4526b0e960a08160023b, 80db8971e5bc4e17a255cb9036b42a47; no voters:
I20260811 02:57:25.479354 11119 raft_consensus.cc:2781] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:25.479563 11119 raft_consensus.cc:686] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [term 1 LEADER]: Becoming Leader. State: Replica: 4dca84a3085d4526b0e960a08160023b, State: Running, Role: LEADER
I20260811 02:57:25.479852 11119 consensus_queue.cc:237] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } }
I20260811 02:57:25.482806 10112 catalog_manager.cc:5495] T 7dfe037a507b42ec899e359c3559789d P 4dca84a3085d4526b0e960a08160023b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dca84a3085d4526b0e960a08160023b (127.8.203.65). New cstate: current_term: 1 leader_uuid: "4dca84a3085d4526b0e960a08160023b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "289bb4b8c9414910bfd26f3b757337fb" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dca84a3085d4526b0e960a08160023b" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80db8971e5bc4e17a255cb9036b42a47" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42011 } health_report { overall_health: UNKNOWN } } }
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[10028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:25.548374 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.540047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.8.203.65:33147)
0811 02:57:25.540186 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.540191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.540325 (+ 134us) server_negotiation.cc:407] Connection header received
0811 02:57:25.540499 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.540501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.540554 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.540646 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:25.541356 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.541861 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.542673 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.542893 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.542982 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.545274 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:25.545311 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:25.545333 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:25.545360 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:25.546744 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.547141 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.547146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.547148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.547227 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.547481 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.547484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.547486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.547735 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:25.547955 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.548106 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.548214 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[10028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417045, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[10028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417045, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:25.574623 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.567189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.8.203.65:33147)
0811 02:57:25.567302 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.567306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.567487 (+ 181us) server_negotiation.cc:407] Connection header received
0811 02:57:25.567592 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.567594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.567633 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.567700 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:25.568356 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.568861 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.569639 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.569860 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.569934 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.572126 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:25.572129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:25.572131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:25.572153 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:25.573427 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.573755 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.573757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.573758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.573822 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.574000 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.574002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.574003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.574139 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:25.574247 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.574415 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.574500 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[10028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[10028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:25.598560 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:25.591267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.8.203.65:33147)
0811 02:57:25.591417 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:25.591421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:25.591497 (+ 76us) server_negotiation.cc:407] Connection header received
0811 02:57:25.591605 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:25.591607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:25.591649 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:25.591710 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:25.592293 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.592770 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.593586 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:25.593821 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:25.593896 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:25.596106 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:25.596110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:25.596112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:25.596136 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:25.597206 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.597584 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.597587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.597588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.597661 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:25.597843 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:25.597845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:25.597846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:25.597994 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:25.598101 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:25.598285 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 02:57:25.598398 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 02:57:25.600118 10336 tablet_service.cc:2908] Rejecting scan request for tablet 7dfe037a507b42ec899e359c3559789d: Uninitialized: clean time has not yet been initialized
I20260811 02:57:25.602065 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10249
I20260811 02:57:25.638087 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10532
I20260811 02:57:25.652171 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10827
I20260811 02:57:25.663174 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10055
2026-08-11T02:57:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2795 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[11150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[11150](info): set up 2 sockets
Aug 11 02:57:25 dist-test-slave-gjw7 krb5kdc[11150](info): commencing operation
krb5kdc: starting...
W20260811 02:57:26.894186 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:26 dist-test-slave-gjw7 krb5kdc[11150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:27.009124 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46053
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:27.120860 11180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:27.121143 11180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:27.121217 11180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:27.125125 11180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:27.125195 11180 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:57:27.125224 11180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:27.125252 11180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:27.125306 11180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:27.125344 11180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:27.130577 11180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46053
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:27.131604 11180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:27.132627 11180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:27.140828 11190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.140861 11187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:27.141010 11180 server_base.cc:1034] running on GCE node
W20260811 02:57:27.140861 11188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:27.141680 11180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:27.142737 11180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:27.143886 11180 hybrid_clock.cc:648] HybridClock initialized: now 1786417047143861 us; error 37 us; skew 500 ppm
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:27.146540 11180 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:27.147349 11180 webserver.cc:460] Webserver started at http://127.8.203.126:44205/ using document root <none> and password file <none>
I20260811 02:57:27.147886 11180 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:27.147951 11180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:27.148156 11180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:27.150182 11180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "72845c1ad9a14dc18955db6b54b7efa4"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "afb0697c5bbac484cf43bd1d41971377"
server_key_iv: "93f592dd0d2b8625c05f01bab39759fc"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.150725 11180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "72845c1ad9a14dc18955db6b54b7efa4"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "afb0697c5bbac484cf43bd1d41971377"
server_key_iv: "93f592dd0d2b8625c05f01bab39759fc"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.154239 11180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:27.156545 11202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.157774 11180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:27.157889 11180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "72845c1ad9a14dc18955db6b54b7efa4"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "afb0697c5bbac484cf43bd1d41971377"
server_key_iv: "93f592dd0d2b8625c05f01bab39759fc"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.157980 11180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:27.167836 11180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:27.170753 11180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:27.170915 11180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:27.178551 11180 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:46053
I20260811 02:57:27.178598 11320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:46053 every 8 connection(s)
I20260811 02:57:27.179430 11180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:27.182050 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:27.184171 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11180
I20260811 02:57:27.184273 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:27.184500 9005 external_mini_cluster.cc:1428] Setting key 859a43567190eeaee56997376bbd395d
I20260811 02:57:27.187981 11321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4: Bootstrap starting.
I20260811 02:57:27.190471 11321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:27.191301 11321 log.cc:824] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:27.193346 11321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4: No bootstrap required, opened a new log
I20260811 02:57:27.197715 11321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72845c1ad9a14dc18955db6b54b7efa4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46053 } }
I20260811 02:57:27.198014 11321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:27.198079 11321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72845c1ad9a14dc18955db6b54b7efa4, State: Initialized, Role: FOLLOWER
I20260811 02:57:27.198580 11321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72845c1ad9a14dc18955db6b54b7efa4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46053 } }
I20260811 02:57:27.198750 11321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:27.198794 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.185726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39520 (local address 127.8.203.126:46053)
0811 02:57:27.186122 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.186142 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.186182 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:57:27.186966 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.186989 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.187399 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.187745 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.188431 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.189882 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.190686 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.191359 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.191513 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.193806 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.193813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.193832 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.193871 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.195752 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.196169 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.196176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.196182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.196332 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.196563 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.196567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.196569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.197093 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.197340 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.197516 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.197819 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260811 02:57:27.198829 11321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:27.198922 11321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:27.199831 11321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72845c1ad9a14dc18955db6b54b7efa4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46053 } }
I20260811 02:57:27.200217 11321 leader_election.cc:302] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72845c1ad9a14dc18955db6b54b7efa4; no voters:
I20260811 02:57:27.200591 11321 leader_election.cc:288] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:27.200685 11327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:27.200912 11327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [term 1 LEADER]: Becoming Leader. State: Replica: 72845c1ad9a14dc18955db6b54b7efa4, State: Running, Role: LEADER
I20260811 02:57:27.201258 11327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72845c1ad9a14dc18955db6b54b7efa4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46053 } }
I20260811 02:57:27.201797 11321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:27.202296 11327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72845c1ad9a14dc18955db6b54b7efa4. Latest consensus state: current_term: 1 leader_uuid: "72845c1ad9a14dc18955db6b54b7efa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72845c1ad9a14dc18955db6b54b7efa4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46053 } } }
I20260811 02:57:27.202486 11327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:27.202939 11338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:27.203524 11330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72845c1ad9a14dc18955db6b54b7efa4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72845c1ad9a14dc18955db6b54b7efa4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46053 } } }
I20260811 02:57:27.203665 11330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:27.206023 11338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:27.211011 11338 catalog_manager.cc:1347] Generated new cluster ID: 63384cf21222478aa7f2ae0dedc80e20
I20260811 02:57:27.211074 11338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:27.221544 11338 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:27.222790 11338 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:27.231367 11338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4: Generated new TSK 0
I20260811 02:57:27.232118 11338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:27.262346 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46053
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:27.371912 11360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:27.372141 11360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:27.372187 11360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:27.376070 11360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:27.376116 11360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:27.376160 11360 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:57:27.376211 11360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:27.381158 11360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:46053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:27.382072 11360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:27.382987 11360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:27.392263 11379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:27.392339 11360 server_base.cc:1034] running on GCE node
W20260811 02:57:27.392248 11377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.392248 11382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:27.392768 11360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:27.393392 11360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:27.394551 11360 hybrid_clock.cc:648] HybridClock initialized: now 1786417047394536 us; error 29 us; skew 500 ppm
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:27.399638 11360 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:27.400460 11360 webserver.cc:460] Webserver started at http://127.8.203.65:39589/ using document root <none> and password file <none>
I20260811 02:57:27.400982 11360 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:27.401046 11360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:27.401250 11360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:27.403398 11360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "758a5ae902f1415c810ad16d347488aa"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "6c763f74dd0f0d6e56078d3f48b04641"
server_key_iv: "b7c34c769de5503f1f0436176317483d"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.403935 11360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "758a5ae902f1415c810ad16d347488aa"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "6c763f74dd0f0d6e56078d3f48b04641"
server_key_iv: "b7c34c769de5503f1f0436176317483d"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.407420 11360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:27.409708 11396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.410828 11360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:27.410943 11360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "758a5ae902f1415c810ad16d347488aa"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "6c763f74dd0f0d6e56078d3f48b04641"
server_key_iv: "b7c34c769de5503f1f0436176317483d"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.411031 11360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:27.428951 11360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:27.431579 11360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:27.431748 11360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:27.433151 11360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:27.433198 11360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.433251 11360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:27.433281 11360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:27.443390 11360 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:45567
I20260811 02:57:27.443671 11620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:45567 every 8 connection(s)
I20260811 02:57:27.444217 11360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:27.446339 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.433991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:54093 (local address 127.8.203.126:46053)
0811 02:57:27.434132 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.434137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.434994 (+ 857us) server_negotiation.cc:407] Connection header received
0811 02:57:27.436100 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.436104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.436161 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.436247 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.437845 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.438476 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.440143 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.440433 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.440515 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.443083 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.443087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.443090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.443119 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.444532 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.445084 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.445088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.445090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.445166 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.445481 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.445483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.445485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.445653 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.445724 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.446121 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.446219 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:57:27.447019 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.434368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46053 (local address 127.8.203.65:54093)
0811 02:57:27.434821 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.434883 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.435850 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.436490 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.436499 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.436942 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:27.437675 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.437697 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.438584 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.438587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.440015 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.440027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.440562 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.440566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.440747 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.441270 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:27.441311 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:27.442927 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:27.444656 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.444661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.444674 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.444987 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.445240 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.445242 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.445243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.445418 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.445724 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:27.445729 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:27.445936 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.446076 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.446298 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":107,"threads_started":1}
I20260811 02:57:27.448151 11621 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46053
I20260811 02:57:27.448290 11621 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:27.448311 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11360
I20260811 02:57:27.448426 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:27.448647 9005 external_mini_cluster.cc:1428] Setting key 465c155ef72527447c2da715629a6c6b
I20260811 02:57:27.448800 11621 heartbeater.cc:511] Master 127.8.203.126:46053 requested a full tablet report, sending...
I20260811 02:57:27.452953 11234 ts_manager.cc:194] Registered new tserver with Master: 758a5ae902f1415c810ad16d347488aa (127.8.203.65:45567)
I20260811 02:57:27.454394 11234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:54093
I20260811 02:57:27.456674 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.450289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:37799 (local address 127.8.203.126:46053)
0811 02:57:27.450389 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.450393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.450600 (+ 207us) server_negotiation.cc:407] Connection header received
0811 02:57:27.450693 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.450695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.450726 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.450784 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.451213 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.451730 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.452615 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.452831 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.452913 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.453906 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.453909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.453912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.453935 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.455255 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.455612 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.455615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.455617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.455692 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.455952 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.455955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.455957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.456117 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.456198 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.456439 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.456534 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 02:57:27.456682 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.450227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46053 (local address 127.8.203.65:37799)
0811 02:57:27.450530 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.450543 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.450631 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.450813 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.450816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.451017 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:27.451129 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.451136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.451851 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.451854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.452509 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.452518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.452924 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.452928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.453013 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.453486 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:27.453500 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:27.453781 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:27.455338 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.455341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.455344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.455527 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.455772 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.455775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.455777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.455862 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.456203 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:27.456206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:27.456291 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.456374 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.456500 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:27.474170 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46053
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:27.598493 11637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:27.598727 11637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:27.598768 11637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:27.602634 11637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:27.602684 11637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:27.602718 11637 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:57:27.602761 11637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:27.607843 11637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:46053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:27.608741 11637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:27.609666 11637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:27.618769 11651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.618777 11649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.618777 11648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:27.619053 11637 server_base.cc:1034] running on GCE node
I20260811 02:57:27.619448 11637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:27.620042 11637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:27.621192 11637 hybrid_clock.cc:648] HybridClock initialized: now 1786417047621167 us; error 36 us; skew 500 ppm
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:27.623646 11637 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:27.624624 11637 webserver.cc:460] Webserver started at http://127.8.203.66:40977/ using document root <none> and password file <none>
I20260811 02:57:27.625232 11637 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:27.625321 11637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:27.625525 11637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:27.627415 11637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "7b4fdb2f146043eca8a54a1a532c1fe2"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "322b3a35f41ca6a6289490a8e587963a"
server_key_iv: "4dd06aa51676b3c02ab2ea5a2d56c1f6"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.627902 11637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "7b4fdb2f146043eca8a54a1a532c1fe2"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "322b3a35f41ca6a6289490a8e587963a"
server_key_iv: "4dd06aa51676b3c02ab2ea5a2d56c1f6"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.631332 11637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:27.633551 11665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.634666 11637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:27.634763 11637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "7b4fdb2f146043eca8a54a1a532c1fe2"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "322b3a35f41ca6a6289490a8e587963a"
server_key_iv: "4dd06aa51676b3c02ab2ea5a2d56c1f6"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.634858 11637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:27.661062 11637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:27.663861 11637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:27.664031 11637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:27.665494 11637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:27.665546 11637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.665601 11637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:27.665629 11637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:27.675783 11637 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:44819
I20260811 02:57:27.676043 11872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:44819 every 8 connection(s)
I20260811 02:57:27.676869 11637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:27.678606 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.666202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:55187 (local address 127.8.203.126:46053)
0811 02:57:27.666346 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.666351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.667220 (+ 869us) server_negotiation.cc:407] Connection header received
0811 02:57:27.668223 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.668227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.668287 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.668345 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.670008 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.670636 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.671739 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.672025 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.672115 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.675050 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.675053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.675056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.675080 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.676640 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.677130 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.677134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.677136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.677239 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.677701 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.677705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.677706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.677902 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.677981 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.678398 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.678489 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:57:27.679381 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.666598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46053 (local address 127.8.203.66:55187)
0811 02:57:27.667058 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.667107 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.667975 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.668753 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.668762 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.669169 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:27.669862 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.669893 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.670756 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.670760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.671621 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.671631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.672193 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.672197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.672400 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.672932 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:27.672952 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:27.674865 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:27.676740 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.676745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.676755 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.677036 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.677414 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.677417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.677419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.677589 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.677996 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:27.678002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:27.678211 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.678362 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.678603 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":109,"threads_started":1}
I20260811 02:57:27.680415 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11637
I20260811 02:57:27.680521 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:27.680609 11875 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46053
I20260811 02:57:27.680734 11875 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:27.680819 9005 external_mini_cluster.cc:1428] Setting key 1801101fde368c8c02beba82cfadbc10
I20260811 02:57:27.681315 11875 heartbeater.cc:511] Master 127.8.203.126:46053 requested a full tablet report, sending...
I20260811 02:57:27.682426 11236 ts_manager.cc:194] Registered new tserver with Master: 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819)
I20260811 02:57:27.683131 11236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:55187
I20260811 02:57:27.689970 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.682809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46053 (local address 127.8.203.66:57585)
0811 02:57:27.683075 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.683089 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.683186 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.683491 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.683494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.683681 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:27.683791 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.683798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.684781 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.684785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.685605 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.685617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.686090 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.686092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.686160 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.686596 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:27.686606 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:27.686899 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:27.688596 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.688599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.688601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.688781 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.689035 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.689037 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.689038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.689102 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.689580 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:27.689582 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:27.689655 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.689732 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.689832 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260811 02:57:27.690060 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.682746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:57585 (local address 127.8.203.126:46053)
0811 02:57:27.682855 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.682858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.683166 (+ 308us) server_negotiation.cc:407] Connection header received
0811 02:57:27.683266 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.683269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.683315 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.683384 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.683919 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.684645 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.685714 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.685993 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.686088 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.687027 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.687031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.687033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.687058 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.688495 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.688866 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.688870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.688871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.688958 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.689176 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.689179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.689180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.689494 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.689599 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.689805 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.689914 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:27.707918 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46053
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:27.815199 11894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:27.815423 11894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:27.815462 11894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:27.819278 11894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:27.819321 11894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:27.819356 11894 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:57:27.819392 11894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:27.824255 11894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:46053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:27.825127 11894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:27.826004 11894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:27.834636 11908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.834662 11905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:27.834664 11906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:27.834748 11894 server_base.cc:1034] running on GCE node
I20260811 02:57:27.835248 11894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:27.835788 11894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:27.836933 11894 hybrid_clock.cc:648] HybridClock initialized: now 1786417047836920 us; error 30 us; skew 500 ppm
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:27.839548 11894 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:27.840360 11894 webserver.cc:460] Webserver started at http://127.8.203.67:36345/ using document root <none> and password file <none>
I20260811 02:57:27.840860 11894 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:27.840931 11894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:27.841127 11894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:27.843091 11894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "4b516d35b39641179b33adc14594b5cf"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "031e2a3d0e74818a887476f4604892f5"
server_key_iv: "a9fcf21fc487119064cc8986590a7468"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.843611 11894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "4b516d35b39641179b33adc14594b5cf"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "031e2a3d0e74818a887476f4604892f5"
server_key_iv: "a9fcf21fc487119064cc8986590a7468"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.847136 11894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:27.849436 11918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.850549 11894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:27.850649 11894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "4b516d35b39641179b33adc14594b5cf"
format_stamp: "Formatted at 2026-08-11 02:57:27 on dist-test-slave-gjw7"
server_key: "031e2a3d0e74818a887476f4604892f5"
server_key_iv: "a9fcf21fc487119064cc8986590a7468"
server_key_version: "encryptionkey@0"
I20260811 02:57:27.850741 11894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:27.864889 11894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:27.867504 11894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:27.867660 11894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:27.869011 11894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:27.869060 11894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:27.869112 11894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:27.869139 11894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:27.879714 11894 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43975
I20260811 02:57:27.879788 12167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43975 every 8 connection(s)
I20260811 02:57:27.880803 11894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:27.883108 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11894
I20260811 02:57:27.883222 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:27.883512 9005 external_mini_cluster.cc:1428] Setting key 29340017245eaba0a25e5cde4a62b8df
I20260811 02:57:27.883700 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.870346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46053 (local address 127.8.203.67:40951)
0811 02:57:27.870826 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.870866 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.871665 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.872346 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.872353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.872653 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:27.873152 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.873170 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.874032 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.874035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.875274 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.875285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.875792 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.875796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.875982 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.876606 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:27.876625 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:27.878287 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:27.880250 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.880256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.880268 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.880777 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.881119 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.881124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.881126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.881402 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.881980 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:27.882003 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:27.882270 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.882467 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.882772 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":132,"threads_started":1}
I20260811 02:57:27.884361 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.869942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:40951 (local address 127.8.203.126:46053)
0811 02:57:27.870132 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.870136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.870965 (+ 829us) server_negotiation.cc:407] Connection header received
0811 02:57:27.871893 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.871897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.871953 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.872007 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.873333 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.873901 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.875387 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.875642 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.875745 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.878461 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.878466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.878468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.878498 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.880135 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.880905 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.880909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.880911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.881011 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.881573 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.881577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.881579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.881876 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.881970 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.884079 (+ 2109us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.884205 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 02:57:27.885619 12168 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46053
I20260811 02:57:27.885756 12168 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:27.886296 12168 heartbeater.cc:511] Master 127.8.203.126:46053 requested a full tablet report, sending...
I20260811 02:57:27.887316 11236 ts_manager.cc:194] Registered new tserver with Master: 4b516d35b39641179b33adc14594b5cf (127.8.203.67:43975)
I20260811 02:57:27.888012 11236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:40951
I20260811 02:57:27.888352 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:57:27.894357 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.887796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:34533 (local address 127.8.203.126:46053)
0811 02:57:27.887904 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.887907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.888325 (+ 418us) server_negotiation.cc:407] Connection header received
0811 02:57:27.888516 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.888520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.888565 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.888637 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.889046 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.889583 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.890345 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.890554 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.890635 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.891569 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.891572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.891575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.891597 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.892830 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.893238 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.893240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.893241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.893338 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.893629 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.893631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.893632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.893830 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.893912 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.894164 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.894253 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:57:27.894456 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.887808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46053 (local address 127.8.203.67:34533)
0811 02:57:27.888253 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.888266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.888390 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.888668 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.888671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.888852 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:27.888944 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.888950 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.889702 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.889705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.890245 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.890253 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.890647 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.890650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.890738 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.891144 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:27.891153 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:27.891450 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:27.892934 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.892938 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.892940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.893159 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.893427 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:27.893431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:27.893432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:27.893542 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:27.893921 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:27.893924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:27.894021 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.894125 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.894263 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":92,"threads_started":1}
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:57:27 dist-test-slave-gjw7 krb5kdc[11150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:27.914778 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.907687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39522 (local address 127.8.203.126:46053)
0811 02:57:27.907805 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.907809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.907820 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:57:27.907866 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.907868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.907905 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.907970 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:27.908670 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.909197 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.910016 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.910223 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.910291 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.912396 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:27.912399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:27.912401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:27.912423 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:27.913542 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.913870 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.913872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.913874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.913936 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:27.914110 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:27.914113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:27.914114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:27.914283 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:27.914386 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.914572 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.914659 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:57:27.917407 11236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:27.920292 11236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:27.934089 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.928896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.8.203.67:43975)
0811 02:57:27.929658 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.929670 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.929693 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:57:27.929763 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.929768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.929947 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.930076 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:27.930282 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.931072 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.932551 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.933389 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.933498 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.933660 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.933859 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.933951 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":98,"threads_started":1}
I20260811 02:57:27.934079 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.928742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43975 (local address 127.0.0.1:35364)
0811 02:57:27.929003 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.929016 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.929090 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.930060 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.930063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.930081 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:27.930181 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.930187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.931175 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.931178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.932416 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.932427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.933505 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.933509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.933618 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.933728 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.933812 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.933933 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":113,"thread_start_us":109,"threads_started":1}
I20260811 02:57:27.934228 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.928191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.8.203.65:45567)
0811 02:57:27.928463 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.928476 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.928498 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:57:27.928655 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.928660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.928866 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.929025 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:27.929890 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.931018 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.932582 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.933398 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.933498 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.933660 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.934007 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.934098 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260811 02:57:27.934451 12201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.928038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45567 (local address 127.0.0.1:60780)
0811 02:57:27.928345 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.928391 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.928509 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.929196 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.929201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.929220 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:27.929384 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.929393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.931132 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.931135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.932477 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.932487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.933712 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.933715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.933787 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.933806 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.933863 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.933934 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":77,"thread_start_us":76,"threads_started":1}
I20260811 02:57:27.934578 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.929545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.8.203.66:44819)
0811 02:57:27.929869 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:27.929882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:27.929906 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:57:27.929979 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:27.929984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:27.930169 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:27.930361 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:27.930530 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.931561 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.932843 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.933444 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.934152 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.934311 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:27.934363 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 02:57:27.934455 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":84,"threads_started":1}
I20260811 02:57:27.936937 11487 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac3a2a04da94076b01450a40ebd3361 (DEFAULT_TABLE table=test-table [id=eb3038df6728406cb0297c41aa142ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:27.936941 12036 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac3a2a04da94076b01450a40ebd3361 (DEFAULT_TABLE table=test-table [id=eb3038df6728406cb0297c41aa142ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:27.937911 11487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac3a2a04da94076b01450a40ebd3361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:27.937938 12036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac3a2a04da94076b01450a40ebd3361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:27.944018 12218 tablet_bootstrap.cc:492] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Bootstrap starting.
I20260811 02:57:27.934082 12210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:27.929071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:44819 (local address 127.0.0.1:45564)
0811 02:57:27.929271 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:27.929284 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:27.929405 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:27.930325 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:27.930328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:27.930346 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:27.930423 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.930428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.931680 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.931684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.932739 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:27.932746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:27.933569 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:27.933572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:27.933672 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:27.933711 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 02:57:27.933793 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:57:27.933923 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":5,"thread_start_us":80,"threads_started":1}
I20260811 02:57:27.944576 12219 tablet_bootstrap.cc:492] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Bootstrap starting.
I20260811 02:57:27.946167 12218 tablet_bootstrap.cc:654] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:27.946491 12219 tablet_bootstrap.cc:654] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:27.947054 12218 log.cc:824] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:27.947160 12219 log.cc:824] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:27.947434 11756 tablet_service.cc:1467] Processing CreateTablet for tablet 5ac3a2a04da94076b01450a40ebd3361 (DEFAULT_TABLE table=test-table [id=eb3038df6728406cb0297c41aa142ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:27.948525 11756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ac3a2a04da94076b01450a40ebd3361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:27.948989 12218 tablet_bootstrap.cc:492] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: No bootstrap required, opened a new log
I20260811 02:57:27.949018 12219 tablet_bootstrap.cc:492] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: No bootstrap required, opened a new log
I20260811 02:57:27.949169 12218 ts_tablet_manager.cc:1397] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:27.949180 12219 ts_tablet_manager.cc:1397] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:57:27.952049 12218 raft_consensus.cc:348] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:27.952248 12218 raft_consensus.cc:374] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:27.952327 12218 raft_consensus.cc:729] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b516d35b39641179b33adc14594b5cf, State: Initialized, Role: FOLLOWER
I20260811 02:57:27.952814 12218 consensus_queue.cc:260] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:27.952864 12219 raft_consensus.cc:348] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:27.953173 12219 raft_consensus.cc:374] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:27.953263 12219 raft_consensus.cc:729] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 758a5ae902f1415c810ad16d347488aa, State: Initialized, Role: FOLLOWER
I20260811 02:57:27.953440 12168 heartbeater.cc:503] Master 127.8.203.126:46053 was elected leader, sending a full tablet report...
I20260811 02:57:27.953735 12218 ts_tablet_manager.cc:1428] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:27.953763 12219 consensus_queue.cc:260] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:27.954489 11621 heartbeater.cc:503] Master 127.8.203.126:46053 was elected leader, sending a full tablet report...
I20260811 02:57:27.954841 12219 ts_tablet_manager.cc:1428] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:57:27.957424 12227 tablet_bootstrap.cc:492] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Bootstrap starting.
I20260811 02:57:27.959095 12227 tablet_bootstrap.cc:654] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:27.959806 12227 log.cc:824] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:27.961570 12227 tablet_bootstrap.cc:492] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: No bootstrap required, opened a new log
I20260811 02:57:27.961736 12227 ts_tablet_manager.cc:1397] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260811 02:57:27.964414 12227 raft_consensus.cc:348] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:27.964613 12227 raft_consensus.cc:374] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:27.964679 12227 raft_consensus.cc:729] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b4fdb2f146043eca8a54a1a532c1fe2, State: Initialized, Role: FOLLOWER
I20260811 02:57:27.965073 12227 consensus_queue.cc:260] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:27.965663 11875 heartbeater.cc:503] Master 127.8.203.126:46053 was elected leader, sending a full tablet report...
I20260811 02:57:27.965902 12227 ts_tablet_manager.cc:1428] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260811 02:57:28.063746 12225 raft_consensus.cc:482] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:28.063956 12225 raft_consensus.cc:504] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:28.065068 12225 leader_election.cc:288] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b516d35b39641179b33adc14594b5cf (127.8.203.67:43975), 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819)
I20260811 02:57:28.069820 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.065828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:56687 (local address 127.8.203.66:44819)
0811 02:57:28.065947 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.065951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.066005 (+ 54us) server_negotiation.cc:407] Connection header received
0811 02:57:28.066116 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.066119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.066158 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.066211 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:28.066443 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.067135 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.068774 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.069264 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.069412 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.069468 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.069629 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.069698 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:57:28.069849 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.065420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:47609 (local address 127.8.203.67:43975)
0811 02:57:28.065547 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.065551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.065620 (+ 69us) server_negotiation.cc:407] Connection header received
0811 02:57:28.065842 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.065845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.065909 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.066000 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:28.066275 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.067065 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.068719 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.069277 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.069417 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.069469 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.069659 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.069730 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:57:28.069862 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.065675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:44819 (local address 127.8.203.65:56687)
0811 02:57:28.065939 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.065950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.066037 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.066240 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.066243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.066260 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:28.066354 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.066358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.067220 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.067223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.068597 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.068609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.069412 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.069416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.069514 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.069537 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.069602 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.069673 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":49,"thread_start_us":86,"threads_started":1}
I20260811 02:57:28.069873 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.065280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43975 (local address 127.8.203.65:47609)
0811 02:57:28.065555 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.065568 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.065676 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.066024 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.066027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.066041 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:28.066170 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.066176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.067168 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.067170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.068602 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.068609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.069417 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.069421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.069543 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.069567 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.069634 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.069706 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":110,"thread_start_us":69,"threads_started":1}
I20260811 02:57:28.070528 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac3a2a04da94076b01450a40ebd3361" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" is_pre_election: true
I20260811 02:57:28.070533 12078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac3a2a04da94076b01450a40ebd3361" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b516d35b39641179b33adc14594b5cf" is_pre_election: true
I20260811 02:57:28.070814 12078 raft_consensus.cc:2454] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 0.
I20260811 02:57:28.070814 11796 raft_consensus.cc:2454] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 0.
I20260811 02:57:28.071271 11406 leader_election.cc:302] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758a5ae902f1415c810ad16d347488aa, 7b4fdb2f146043eca8a54a1a532c1fe2; no voters:
I20260811 02:57:28.071527 12225 raft_consensus.cc:2781] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:28.071601 12225 raft_consensus.cc:482] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:28.071630 12225 raft_consensus.cc:3037] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.072615 12225 raft_consensus.cc:504] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:28.072921 12225 leader_election.cc:288] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 election: Requested vote from peers 4b516d35b39641179b33adc14594b5cf (127.8.203.67:43975), 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819)
I20260811 02:57:28.073240 12078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac3a2a04da94076b01450a40ebd3361" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b516d35b39641179b33adc14594b5cf"
I20260811 02:57:28.073313 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ac3a2a04da94076b01450a40ebd3361" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2"
I20260811 02:57:28.073387 12078 raft_consensus.cc:3037] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.073426 11796 raft_consensus.cc:3037] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.074429 11796 raft_consensus.cc:2454] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 1.
I20260811 02:57:28.074689 11406 leader_election.cc:302] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758a5ae902f1415c810ad16d347488aa, 7b4fdb2f146043eca8a54a1a532c1fe2; no voters:
I20260811 02:57:28.074689 12078 raft_consensus.cc:2454] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 1.
I20260811 02:57:28.074873 12225 raft_consensus.cc:2781] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:28.075083 12225 raft_consensus.cc:686] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 1 LEADER]: Becoming Leader. State: Replica: 758a5ae902f1415c810ad16d347488aa, State: Running, Role: LEADER
I20260811 02:57:28.075402 12225 consensus_queue.cc:237] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } }
I20260811 02:57:28.078457 11236 catalog_manager.cc:5495] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 758a5ae902f1415c810ad16d347488aa (127.8.203.65). New cstate: current_term: 1 leader_uuid: "758a5ae902f1415c810ad16d347488aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260811 02:57:28.124784 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.121661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.8.203.65:45567)
0811 02:57:28.121773 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.121776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.121789 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:28.121992 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.121994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.122051 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.122131 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:28.122323 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.122812 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.123582 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.123782 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.123846 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.124089 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:28.124476 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:28.124554 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.124606 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.124673 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 02:57:28.130330 12078 raft_consensus.cc:1261] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Refusing update from remote peer 758a5ae902f1415c810ad16d347488aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:28.130650 11796 raft_consensus.cc:1261] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Refusing update from remote peer 758a5ae902f1415c810ad16d347488aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:28.131023 12225 consensus_queue.cc:1035] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:28.131220 12233 consensus_queue.cc:1035] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:57:28.131986 12169 tablet.cc:2369] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:28.137573 12238 mvcc.cc:205] Tried to move back new op lower bound from 7317164229134778368 to 7317164228919853056. Current Snapshot: MvccSnapshot[applied={T|T < 7317164229134778368}]
I20260811 02:57:28.139951 12241 mvcc.cc:205] Tried to move back new op lower bound from 7317164229134778368 to 7317164228919853056. Current Snapshot: MvccSnapshot[applied={T|T < 7317164229134778368 or (T in {7317164229134778368})}]
I20260811 02:57:28.148949 11234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:28.149180 11234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:28.151561 11234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4: Sending DeleteTablet for 3 replicas of tablet 5ac3a2a04da94076b01450a40ebd3361
I20260811 02:57:28.152283 11487 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac3a2a04da94076b01450a40ebd3361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:28 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:60780
I20260811 02:57:28.152336 12036 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac3a2a04da94076b01450a40ebd3361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:28 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:35364
I20260811 02:57:28.152462 11756 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ac3a2a04da94076b01450a40ebd3361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:28 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:45564
I20260811 02:57:28.152788 12253 tablet_replica.cc:330] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: stopping tablet replica
I20260811 02:57:28.152833 12257 tablet_replica.cc:330] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: stopping tablet replica
I20260811 02:57:28.153126 12257 raft_consensus.cc:2229] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:28.153154 12253 raft_consensus.cc:2229] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:28.153640 12256 tablet_replica.cc:330] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: stopping tablet replica
I20260811 02:57:28.153775 11234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:28.153901 12256 raft_consensus.cc:2229] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:28.154091 12253 raft_consensus.cc:2258] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:28.154150 12256 raft_consensus.cc:2258] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 02:57:28.154166 11234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:28.153458 12257 raft_consensus.cc:2258] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:28.155927 12257 ts_tablet_manager.cc:1905] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:28.156987 12253 ts_tablet_manager.cc:1905] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:28.155941 12256 ts_tablet_manager.cc:1905] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:28.158763 12257 ts_tablet_manager.cc:1918] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:28.158882 12257 log.cc:1196] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/5ac3a2a04da94076b01450a40ebd3361
I20260811 02:57:28.159197 12257 ts_tablet_manager.cc:1939] T 5ac3a2a04da94076b01450a40ebd3361 P 7b4fdb2f146043eca8a54a1a532c1fe2: Deleting consensus metadata
I20260811 02:57:28.160226 11487 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcf2f5655314814a5d64f00c08ff1b8 (DEFAULT_TABLE table=test-table [id=7b17909bea7d4f1ea5924ab05b3b4a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:28.160488 11487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcf2f5655314814a5d64f00c08ff1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:28.161687 12256 ts_tablet_manager.cc:1918] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:28.161777 12256 log.cc:1196] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/5ac3a2a04da94076b01450a40ebd3361
I20260811 02:57:28.162074 12256 ts_tablet_manager.cc:1939] T 5ac3a2a04da94076b01450a40ebd3361 P 4b516d35b39641179b33adc14594b5cf: Deleting consensus metadata
I20260811 02:57:28.162487 12219 tablet_bootstrap.cc:492] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Bootstrap starting.
I20260811 02:57:28.162688 11756 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcf2f5655314814a5d64f00c08ff1b8 (DEFAULT_TABLE table=test-table [id=7b17909bea7d4f1ea5924ab05b3b4a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:28.162911 11756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcf2f5655314814a5d64f00c08ff1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:28.163425 12219 tablet_bootstrap.cc:654] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:28.163784 12253 ts_tablet_manager.cc:1918] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:28.163867 12253 log.cc:1196] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/5ac3a2a04da94076b01450a40ebd3361
I20260811 02:57:28.164160 12253 ts_tablet_manager.cc:1939] T 5ac3a2a04da94076b01450a40ebd3361 P 758a5ae902f1415c810ad16d347488aa: Deleting consensus metadata
I20260811 02:57:28.164870 12227 tablet_bootstrap.cc:492] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Bootstrap starting.
I20260811 02:57:28.165050 12036 tablet_service.cc:1467] Processing CreateTablet for tablet 3bcf2f5655314814a5d64f00c08ff1b8 (DEFAULT_TABLE table=test-table [id=7b17909bea7d4f1ea5924ab05b3b4a75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:28.165271 12036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bcf2f5655314814a5d64f00c08ff1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:28.165786 12227 tablet_bootstrap.cc:654] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:28.166999 12227 tablet_bootstrap.cc:492] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: No bootstrap required, opened a new log
I20260811 02:57:28.167088 12227 ts_tablet_manager.cc:1397] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 02:57:28.167405 12218 tablet_bootstrap.cc:492] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: Bootstrap starting.
I20260811 02:57:28.167522 12227 raft_consensus.cc:348] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.167629 12227 raft_consensus.cc:374] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:28.167662 12227 raft_consensus.cc:729] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b4fdb2f146043eca8a54a1a532c1fe2, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.167783 12227 consensus_queue.cc:260] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.167914 11209 catalog_manager.cc:4841] TS 758a5ae902f1415c810ad16d347488aa (127.8.203.65:45567): tablet 5ac3a2a04da94076b01450a40ebd3361 (table test-table [id=eb3038df6728406cb0297c41aa142ff7]) successfully deleted
I20260811 02:57:28.168045 12227 ts_tablet_manager.cc:1428] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:57:28.168077 11209 catalog_manager.cc:4841] TS 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819): tablet 5ac3a2a04da94076b01450a40ebd3361 (table test-table [id=eb3038df6728406cb0297c41aa142ff7]) successfully deleted
I20260811 02:57:28.168229 11209 catalog_manager.cc:4841] TS 4b516d35b39641179b33adc14594b5cf (127.8.203.67:43975): tablet 5ac3a2a04da94076b01450a40ebd3361 (table test-table [id=eb3038df6728406cb0297c41aa142ff7]) successfully deleted
I20260811 02:57:28.168287 12218 tablet_bootstrap.cc:654] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:28.169633 12218 tablet_bootstrap.cc:492] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: No bootstrap required, opened a new log
I20260811 02:57:28.169718 12218 ts_tablet_manager.cc:1397] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:28.170152 12218 raft_consensus.cc:348] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.170262 12218 raft_consensus.cc:374] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:28.170295 12218 raft_consensus.cc:729] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b516d35b39641179b33adc14594b5cf, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.170405 12218 consensus_queue.cc:260] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.170630 12218 ts_tablet_manager.cc:1428] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:28.171798 12219 tablet_bootstrap.cc:492] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: No bootstrap required, opened a new log
I20260811 02:57:28.171895 12219 ts_tablet_manager.cc:1397] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20260811 02:57:28.172272 12219 raft_consensus.cc:348] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.172369 12219 raft_consensus.cc:374] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:28.172390 12219 raft_consensus.cc:729] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 758a5ae902f1415c810ad16d347488aa, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.172508 12219 consensus_queue.cc:260] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.172730 12219 ts_tablet_manager.cc:1428] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260811 02:57:28.177973 11876 tablet.cc:2369] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:28.195439 11622 tablet.cc:2369] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:28.233466 12233 raft_consensus.cc:482] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:28.233596 12233 raft_consensus.cc:504] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.234025 12233 leader_election.cc:288] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819), 4b516d35b39641179b33adc14594b5cf (127.8.203.67:43975)
I20260811 02:57:28.234464 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcf2f5655314814a5d64f00c08ff1b8" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" is_pre_election: true
I20260811 02:57:28.234485 12078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcf2f5655314814a5d64f00c08ff1b8" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b516d35b39641179b33adc14594b5cf" is_pre_election: true
I20260811 02:57:28.234606 11796 raft_consensus.cc:2454] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 0.
I20260811 02:57:28.234633 12078 raft_consensus.cc:2454] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 0.
I20260811 02:57:28.234889 11406 leader_election.cc:302] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 758a5ae902f1415c810ad16d347488aa, 7b4fdb2f146043eca8a54a1a532c1fe2; no voters:
I20260811 02:57:28.235067 12233 raft_consensus.cc:2781] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:28.235132 12233 raft_consensus.cc:482] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:28.235179 12233 raft_consensus.cc:3037] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.236120 12233 raft_consensus.cc:504] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.236431 12233 leader_election.cc:288] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 election: Requested vote from peers 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819), 4b516d35b39641179b33adc14594b5cf (127.8.203.67:43975)
I20260811 02:57:28.236802 12078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcf2f5655314814a5d64f00c08ff1b8" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b516d35b39641179b33adc14594b5cf"
I20260811 02:57:28.236809 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bcf2f5655314814a5d64f00c08ff1b8" candidate_uuid: "758a5ae902f1415c810ad16d347488aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2"
I20260811 02:57:28.236927 12078 raft_consensus.cc:3037] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.236927 11796 raft_consensus.cc:3037] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.238049 12078 raft_consensus.cc:2454] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 1.
I20260811 02:57:28.238049 11796 raft_consensus.cc:2454] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 758a5ae902f1415c810ad16d347488aa in term 1.
I20260811 02:57:28.238322 11406 leader_election.cc:302] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b516d35b39641179b33adc14594b5cf, 758a5ae902f1415c810ad16d347488aa; no voters:
I20260811 02:57:28.238477 12233 raft_consensus.cc:2781] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:28.238574 12233 raft_consensus.cc:686] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 1 LEADER]: Becoming Leader. State: Replica: 758a5ae902f1415c810ad16d347488aa, State: Running, Role: LEADER
I20260811 02:57:28.238722 12233 consensus_queue.cc:237] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.240552 11236 catalog_manager.cc:5495] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 758a5ae902f1415c810ad16d347488aa (127.8.203.65). New cstate: current_term: 1 leader_uuid: "758a5ae902f1415c810ad16d347488aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } health_report { overall_health: UNKNOWN } } }
Aug 11 02:57:28 dist-test-slave-gjw7 krb5kdc[11150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:28.287914 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.278959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.8.203.126:46053)
0811 02:57:28.279098 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.279103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.279244 (+ 141us) server_negotiation.cc:407] Connection header received
0811 02:57:28.279344 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.279347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.279396 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.279487 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:28.280084 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.281003 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.281874 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.282255 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.282368 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.284623 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:28.284626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:28.284628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:28.284649 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:28.286059 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:28.286636 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:28.286639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:28.286640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:28.286710 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:28.286998 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:28.287000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:28.287001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:28.287164 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:28.287235 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.287561 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.287756 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":60}
I20260811 02:57:28.287932 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.278874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46053 (local address 127.0.0.1:39532)
0811 02:57:28.279160 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.279174 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.279274 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.279505 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.279507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.279886 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:28.279983 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.279989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.281120 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.281122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.281731 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.281742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.282368 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.282370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.282433 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.282996 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:28.283007 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:28.284487 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:28.286163 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:28.286183 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:28.286190 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:28.286537 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:28.286787 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:28.286790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:28.286792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:28.286918 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:28.287255 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:28.287262 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:28.287399 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.287520 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.287697 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":83,"threads_started":1}
I20260811 02:57:28.291859 11236 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:39532:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:57:28.292567 11236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:28.297787 11487 tablet_service.cc:1467] Processing CreateTablet for tablet 3a26d2bd844f49cf8cacaea95fd25b15 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc86e72542a94e9e93b2a6771d6f10cb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:57:28.297787 11756 tablet_service.cc:1467] Processing CreateTablet for tablet 3a26d2bd844f49cf8cacaea95fd25b15 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc86e72542a94e9e93b2a6771d6f10cb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:57:28.297963 12036 tablet_service.cc:1467] Processing CreateTablet for tablet 3a26d2bd844f49cf8cacaea95fd25b15 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fc86e72542a94e9e93b2a6771d6f10cb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:57:28.298182 11487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a26d2bd844f49cf8cacaea95fd25b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:28.298182 11756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a26d2bd844f49cf8cacaea95fd25b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:28.298348 12036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a26d2bd844f49cf8cacaea95fd25b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:28.301434 12218 tablet_bootstrap.cc:492] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: Bootstrap starting.
I20260811 02:57:28.302193 12218 tablet_bootstrap.cc:654] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:28.303016 12227 tablet_bootstrap.cc:492] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: Bootstrap starting.
I20260811 02:57:28.303737 12227 tablet_bootstrap.cc:654] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:28.304826 12227 tablet_bootstrap.cc:492] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: No bootstrap required, opened a new log
I20260811 02:57:28.304924 12227 ts_tablet_manager.cc:1397] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 02:57:28.305338 12227 raft_consensus.cc:348] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.305442 12227 raft_consensus.cc:374] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:28.305472 12227 raft_consensus.cc:729] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b4fdb2f146043eca8a54a1a532c1fe2, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.305595 12227 consensus_queue.cc:260] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.305860 12227 ts_tablet_manager.cc:1428] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:28.305927 12219 tablet_bootstrap.cc:492] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: Bootstrap starting.
I20260811 02:57:28.306135 12230 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.306272 12230 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:28.306746 12219 tablet_bootstrap.cc:654] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:28.307585 12218 tablet_bootstrap.cc:492] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: No bootstrap required, opened a new log
I20260811 02:57:28.307644 12218 ts_tablet_manager.cc:1397] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260811 02:57:28.307932 12219 tablet_bootstrap.cc:492] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: No bootstrap required, opened a new log
I20260811 02:57:28.308003 12219 ts_tablet_manager.cc:1397] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:28.307965 12218 raft_consensus.cc:348] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.308038 12218 raft_consensus.cc:374] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:28.308058 12218 raft_consensus.cc:729] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b516d35b39641179b33adc14594b5cf, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.308152 12218 consensus_queue.cc:260] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.308393 12218 ts_tablet_manager.cc:1428] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:57:28.308341 12219 raft_consensus.cc:348] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.308460 12219 raft_consensus.cc:374] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:28.308372 12224 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.308495 12219 raft_consensus.cc:729] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 758a5ae902f1415c810ad16d347488aa, State: Initialized, Role: FOLLOWER
I20260811 02:57:28.308557 12224 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:28.308621 12219 consensus_queue.cc:260] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.308928 12219 ts_tablet_manager.cc:1428] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:28.308902 12233 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.309019 12233 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:28.389591 12224 raft_consensus.cc:482] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:28.389716 12224 raft_consensus.cc:504] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.390726 12224 leader_election.cc:288] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 758a5ae902f1415c810ad16d347488aa (127.8.203.65:45567), 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819)
I20260811 02:57:28.394474 12289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.390915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45567 (local address 127.8.203.67:58361)
0811 02:57:28.391147 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.391160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.391271 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.391546 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.391549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.391566 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:28.391689 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.391695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.392521 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.392524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.393416 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.393426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.394064 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.394068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.394152 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.394180 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.394245 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.394309 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":60,"threads_started":1}
I20260811 02:57:28.394496 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.390995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:58361 (local address 127.8.203.65:45567)
0811 02:57:28.391156 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.391161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.391233 (+ 72us) server_negotiation.cc:407] Connection header received
0811 02:57:28.391386 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.391390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.391447 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.391519 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:28.391808 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.392427 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.393519 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.393957 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.394050 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.394091 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.394285 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.394370 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:57:28.394560 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.391359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:45105 (local address 127.8.203.66:44819)
0811 02:57:28.391490 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.391494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.391600 (+ 106us) server_negotiation.cc:407] Connection header received
0811 02:57:28.391697 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.391700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.391746 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.391810 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:28.392071 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.392586 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.393679 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.394069 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.394171 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.394215 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.394392 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.394457 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:57:28.394747 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.391242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:44819 (local address 127.8.203.67:45105)
0811 02:57:28.391516 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.391529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.391620 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.391847 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.391850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.391866 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:28.391969 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.391976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.392692 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.392695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.393556 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.393566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.394183 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.394186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.394274 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.394299 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.394367 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.394427 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":73,"threads_started":1}
I20260811 02:57:28.395030 11524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26d2bd844f49cf8cacaea95fd25b15" candidate_uuid: "4b516d35b39641179b33adc14594b5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "758a5ae902f1415c810ad16d347488aa" is_pre_election: true
I20260811 02:57:28.395193 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26d2bd844f49cf8cacaea95fd25b15" candidate_uuid: "4b516d35b39641179b33adc14594b5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" is_pre_election: true
I20260811 02:57:28.395242 11524 raft_consensus.cc:2454] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b516d35b39641179b33adc14594b5cf in term 0.
I20260811 02:57:28.395326 11796 raft_consensus.cc:2454] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b516d35b39641179b33adc14594b5cf in term 0.
I20260811 02:57:28.395574 11931 leader_election.cc:302] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b516d35b39641179b33adc14594b5cf, 758a5ae902f1415c810ad16d347488aa; no voters:
I20260811 02:57:28.395780 12224 raft_consensus.cc:2781] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:28.395833 12224 raft_consensus.cc:482] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:28.395872 12224 raft_consensus.cc:3037] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.396816 12224 raft_consensus.cc:504] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.397121 12224 leader_election.cc:288] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [CANDIDATE]: Term 1 election: Requested vote from peers 758a5ae902f1415c810ad16d347488aa (127.8.203.65:45567), 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819)
I20260811 02:57:28.397449 11524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26d2bd844f49cf8cacaea95fd25b15" candidate_uuid: "4b516d35b39641179b33adc14594b5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "758a5ae902f1415c810ad16d347488aa"
I20260811 02:57:28.397500 11796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a26d2bd844f49cf8cacaea95fd25b15" candidate_uuid: "4b516d35b39641179b33adc14594b5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2"
I20260811 02:57:28.397557 11524 raft_consensus.cc:3037] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.397590 11796 raft_consensus.cc:3037] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:28.398411 11796 raft_consensus.cc:2454] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b516d35b39641179b33adc14594b5cf in term 1.
I20260811 02:57:28.398456 11524 raft_consensus.cc:2454] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b516d35b39641179b33adc14594b5cf in term 1.
I20260811 02:57:28.398746 11931 leader_election.cc:302] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b516d35b39641179b33adc14594b5cf, 758a5ae902f1415c810ad16d347488aa; no voters:
I20260811 02:57:28.398900 12224 raft_consensus.cc:2781] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:28.399065 12224 raft_consensus.cc:686] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [term 1 LEADER]: Becoming Leader. State: Replica: 4b516d35b39641179b33adc14594b5cf, State: Running, Role: LEADER
I20260811 02:57:28.399286 12224 consensus_queue.cc:237] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } }
I20260811 02:57:28.400489 12291 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: TxnStatusTablet state changed. Reason: New leader 4b516d35b39641179b33adc14594b5cf. Latest consensus state: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.400596 12291 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:57:28.400913 12293 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:57:28.401115 12293 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:57:28.401690 11236 catalog_manager.cc:5495] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b516d35b39641179b33adc14594b5cf (127.8.203.67). New cstate: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:28.434018 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.430352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.8.203.67:43975)
0811 02:57:28.430463 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.430466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.430478 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:57:28.430675 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.430677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.430725 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.430802 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:28.431046 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.431514 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.432671 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.432869 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.432939 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.433229 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:28.433672 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:28.433754 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.433870 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.433916 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:57:28.434083 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.430245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43975 (local address 127.0.0.1:35370)
0811 02:57:28.430422 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.430440 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.430546 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.430829 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.430832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.430848 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:28.430956 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.430962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.431600 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.431603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.432573 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.432583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.432959 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.432963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.433058 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.433111 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:28.433759 (+ 648us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:28.433774 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.433860 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.433945 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 02:57:28.436829 11796 raft_consensus.cc:1261] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Refusing update from remote peer 4b516d35b39641179b33adc14594b5cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:28.437023 11524 raft_consensus.cc:1261] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Refusing update from remote peer 4b516d35b39641179b33adc14594b5cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:28.437325 12291 consensus_queue.cc:1035] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:28.437578 12224 consensus_queue.cc:1035] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:28.438712 12230 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: TxnStatusTablet state changed. Reason: New leader 4b516d35b39641179b33adc14594b5cf. Latest consensus state: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.438822 12230 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:28.440178 12233 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: TxnStatusTablet state changed. Reason: New leader 4b516d35b39641179b33adc14594b5cf. Latest consensus state: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.440292 12233 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:28.441439 12224 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.441540 12224 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:57:28.441636 12242 mvcc.cc:205] Tried to move back new op lower bound from 7317164230392512512 to 7317164230245724160. Current Snapshot: MvccSnapshot[applied={T|T < 7317164230392512512}]
I20260811 02:57:28.443327 12291 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.443434 12291 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 4b516d35b39641179b33adc14594b5cf: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:57:28.446733 12233 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.446846 12233 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 758a5ae902f1415c810ad16d347488aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:28.449410 12230 tablet_replica.cc:439] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b516d35b39641179b33adc14594b5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "758a5ae902f1415c810ad16d347488aa" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45567 } } peers { permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 } } peers { permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 } } }
I20260811 02:57:28.449520 12230 tablet_replica.cc:442] T 3a26d2bd844f49cf8cacaea95fd25b15 P 7b4fdb2f146043eca8a54a1a532c1fe2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:28.465205 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.461744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:34507 (local address 127.8.203.67:43975)
0811 02:57:28.461866 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.461883 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.461896 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:28.461938 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.461941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.461986 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.462054 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:28.462283 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.463063 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.464269 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.464493 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.464572 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.464858 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:28.464921 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:28.464965 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.465062 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.465105 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:57:28.465278 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.461411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43975 (local address 127.8.203.65:34507)
0811 02:57:28.461708 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.461722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.461817 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.462070 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.462073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.462080 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:28.462187 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.462193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.463168 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.463172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.464140 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.464152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.464591 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.464594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.464683 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.464736 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:28.464984 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:28.464990 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.465055 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.465135 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
I20260811 02:57:28.471536 12078 raft_consensus.cc:1261] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Refusing update from remote peer 758a5ae902f1415c810ad16d347488aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:28.471582 11796 raft_consensus.cc:1261] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Refusing update from remote peer 758a5ae902f1415c810ad16d347488aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:28.471946 12225 consensus_queue.cc:1035] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b4fdb2f146043eca8a54a1a532c1fe2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:28.472110 12233 consensus_queue.cc:1035] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b516d35b39641179b33adc14594b5cf" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:28.494141 12313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.490104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45567 (local address 127.8.203.67:37107)
0811 02:57:28.490427 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:28.490443 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:28.490540 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:28.491081 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:28.491085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:28.491094 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:28.491371 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.491381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.492216 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.492219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.492862 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:28.492879 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.493317 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.493321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:28.493405 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.493475 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:28.493851 (+ 376us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:28.493857 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:57:28.493915 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 02:57:28.493993 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":113,"threads_started":1}
I20260811 02:57:28.494127 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.490385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:37107 (local address 127.8.203.65:45567)
0811 02:57:28.490490 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.490495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.490508 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:28.490790 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.490793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.490851 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.490935 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:28.491506 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.492108 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.492977 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.493202 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.493309 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.493634 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:28.493747 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:28.493813 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.493939 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.493992 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:57:28.518468 11234 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:28.518615 11234 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:28.520545 11234 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72845c1ad9a14dc18955db6b54b7efa4: Sending DeleteTablet for 3 replicas of tablet 3bcf2f5655314814a5d64f00c08ff1b8
I20260811 02:57:28.521049 11487 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bcf2f5655314814a5d64f00c08ff1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:28 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:60780
I20260811 02:57:28.521183 11756 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bcf2f5655314814a5d64f00c08ff1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:28 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:45564
I20260811 02:57:28.521224 12253 tablet_replica.cc:330] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: stopping tablet replica
I20260811 02:57:28.521359 12253 raft_consensus.cc:2229] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:28.521387 12257 tablet_replica.cc:330] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: stopping tablet replica
I20260811 02:57:28.521489 12257 raft_consensus.cc:2229] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:28.521523 12253 raft_consensus.cc:2258] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:28.521586 12257 raft_consensus.cc:2258] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:28.522312 12253 ts_tablet_manager.cc:1905] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:28.522330 12257 ts_tablet_manager.cc:1905] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:28.525151 12257 ts_tablet_manager.cc:1918] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:28.525223 12257 log.cc:1196] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/3bcf2f5655314814a5d64f00c08ff1b8
I20260811 02:57:28.525238 12253 ts_tablet_manager.cc:1918] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:28.525365 12253 log.cc:1196] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/3bcf2f5655314814a5d64f00c08ff1b8
I20260811 02:57:28.525490 12257 ts_tablet_manager.cc:1939] T 3bcf2f5655314814a5d64f00c08ff1b8 P 7b4fdb2f146043eca8a54a1a532c1fe2: Deleting consensus metadata
I20260811 02:57:28.525610 12253 ts_tablet_manager.cc:1939] T 3bcf2f5655314814a5d64f00c08ff1b8 P 758a5ae902f1415c810ad16d347488aa: Deleting consensus metadata
I20260811 02:57:28.526127 11209 catalog_manager.cc:4841] TS 7b4fdb2f146043eca8a54a1a532c1fe2 (127.8.203.66:44819): tablet 3bcf2f5655314814a5d64f00c08ff1b8 (table test-table [id=7b17909bea7d4f1ea5924ab05b3b4a75]) successfully deleted
I20260811 02:57:28.526296 11209 catalog_manager.cc:4841] TS 758a5ae902f1415c810ad16d347488aa (127.8.203.65:45567): tablet 3bcf2f5655314814a5d64f00c08ff1b8 (table test-table [id=7b17909bea7d4f1ea5924ab05b3b4a75]) successfully deleted
I20260811 02:57:28.526600 12036 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bcf2f5655314814a5d64f00c08ff1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:28 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:35364
I20260811 02:57:28.526798 12256 tablet_replica.cc:330] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: stopping tablet replica
I20260811 02:57:28.526919 12256 raft_consensus.cc:2229] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:28.527037 12256 raft_consensus.cc:2258] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:28.527760 12256 ts_tablet_manager.cc:1905] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:28.529837 12256 ts_tablet_manager.cc:1918] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:28.529898 12256 log.cc:1196] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/3bcf2f5655314814a5d64f00c08ff1b8
I20260811 02:57:28.530066 12256 ts_tablet_manager.cc:1939] T 3bcf2f5655314814a5d64f00c08ff1b8 P 4b516d35b39641179b33adc14594b5cf: Deleting consensus metadata
Aug 11 02:57:28 dist-test-slave-gjw7 krb5kdc[11150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417047, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:28.530622 11209 catalog_manager.cc:4841] TS 4b516d35b39641179b33adc14594b5cf (127.8.203.67:43975): tablet 3bcf2f5655314814a5d64f00c08ff1b8 (table test-table [id=7b17909bea7d4f1ea5924ab05b3b4a75]) successfully deleted
I20260811 02:57:28.534317 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.522728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.8.203.65:45567)
0811 02:57:28.522830 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.522835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.523345 (+ 510us) server_negotiation.cc:407] Connection header received
0811 02:57:28.523533 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.523536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.523589 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.523638 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:28.524611 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.525353 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.526671 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.526973 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.527074 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.531091 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:28.531101 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:28.531113 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:28.531135 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:28.532652 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:28.533116 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:28.533121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:28.533122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:28.533196 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:28.533491 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:28.533493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:28.533495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:28.533728 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:28.533882 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.534103 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.534198 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 02:57:28.534685 11573 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60786
I20260811 02:57:28.542917 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:28.536249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39538 (local address 127.8.203.126:46053)
0811 02:57:28.536364 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:28.536368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:28.536479 (+ 111us) server_negotiation.cc:407] Connection header received
0811 02:57:28.536585 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:28.536587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:28.536632 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:28.536694 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:28.537340 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.537858 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.538652 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:28.538883 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:28.538966 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:28.540186 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:28.540189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:28.540198 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:28.540223 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:28.541480 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:28.541838 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:28.541841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:28.541843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:28.541924 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:28.542160 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:28.542163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:28.542165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:28.542329 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:28.542439 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:28.542658 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 02:57:28.542761 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 02:57:28.543290 11234 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39538
I20260811 02:57:28.544426 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11360
I20260811 02:57:28.588940 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11637
I20260811 02:57:28.615664 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11894
I20260811 02:57:28.630475 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11180
2026-08-11T02:57:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2967 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:28 dist-test-slave-gjw7 krb5kdc[12331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:28 dist-test-slave-gjw7 krb5kdc[12331](info): set up 2 sockets
Aug 11 02:57:28 dist-test-slave-gjw7 krb5kdc[12331](info): commencing operation
krb5kdc: starting...
W20260811 02:57:29.828549 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:29 dist-test-slave-gjw7 krb5kdc[12331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:29.944416 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46701
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:30.055172 12356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:30.055418 12356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:30.055472 12356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:30.059355 12356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:30.059401 12356 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:57:30.059424 12356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:30.059448 12356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:30.059469 12356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:30.059497 12356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:30.064492 12356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46701
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:30.065485 12356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:30.066438 12356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:30.074626 12364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.074654 12363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.074626 12367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.074723 12356 server_base.cc:1034] running on GCE node
I20260811 02:57:30.075469 12356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:30.076565 12356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:30.077718 12356 hybrid_clock.cc:648] HybridClock initialized: now 1786417050077703 us; error 25 us; skew 500 ppm
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:30.080287 12356 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:30.081139 12356 webserver.cc:460] Webserver started at http://127.8.203.126:43903/ using document root <none> and password file <none>
I20260811 02:57:30.081688 12356 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:30.081749 12356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:30.081925 12356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:30.083846 12356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "20ec90b4e9d3466a982fc0efc96426d4"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "bdb8d9ca6d3f7f3677fc95c84831da02"
server_key_iv: "e6a00a144d76c5d322492aed706edd42"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.084355 12356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "20ec90b4e9d3466a982fc0efc96426d4"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "bdb8d9ca6d3f7f3677fc95c84831da02"
server_key_iv: "e6a00a144d76c5d322492aed706edd42"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.087896 12356 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:30.090180 12377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.091351 12356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:30.091449 12356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "20ec90b4e9d3466a982fc0efc96426d4"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "bdb8d9ca6d3f7f3677fc95c84831da02"
server_key_iv: "e6a00a144d76c5d322492aed706edd42"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.091531 12356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:30.128513 12356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:30.131448 12356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:30.131605 12356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:30.139159 12356 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:46701
I20260811 02:57:30.139176 12500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:46701 every 8 connection(s)
I20260811 02:57:30.140148 12356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:30.142904 12505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:30.148281 12505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Bootstrap starting.
I20260811 02:57:30.149771 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 12356
I20260811 02:57:30.149878 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:30.150117 9005 external_mini_cluster.cc:1428] Setting key 9792f3e04715551c5dd6bfe2621bf028
I20260811 02:57:30.150868 12505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:30.151707 12505 log.cc:824] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:30.153901 12505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: No bootstrap required, opened a new log
I20260811 02:57:30.158082 12505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ec90b4e9d3466a982fc0efc96426d4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46701 } }
I20260811 02:57:30.158313 12505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:30.158349 12505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ec90b4e9d3466a982fc0efc96426d4, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.158787 12505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ec90b4e9d3466a982fc0efc96426d4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46701 } }
I20260811 02:57:30.158926 12505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:30.158993 12505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:30.159092 12505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:30.160091 12505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ec90b4e9d3466a982fc0efc96426d4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46701 } }
I20260811 02:57:30.160475 12505 leader_election.cc:302] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20ec90b4e9d3466a982fc0efc96426d4; no voters:
I20260811 02:57:30.160851 12505 leader_election.cc:288] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:30.160979 12512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:30.161268 12512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [term 1 LEADER]: Becoming Leader. State: Replica: 20ec90b4e9d3466a982fc0efc96426d4, State: Running, Role: LEADER
I20260811 02:57:30.161633 12512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ec90b4e9d3466a982fc0efc96426d4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46701 } }
I20260811 02:57:30.162153 12505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:30.163427 12513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20ec90b4e9d3466a982fc0efc96426d4. Latest consensus state: current_term: 1 leader_uuid: "20ec90b4e9d3466a982fc0efc96426d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ec90b4e9d3466a982fc0efc96426d4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46701 } } }
I20260811 02:57:30.163883 12513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:30.163760 12515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20ec90b4e9d3466a982fc0efc96426d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ec90b4e9d3466a982fc0efc96426d4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46701 } } }
I20260811 02:57:30.164482 12515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:30.165634 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.151179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.8.203.126:46701)
0811 02:57:30.151657 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.151677 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.151721 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:57:30.152472 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.152498 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.152908 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.153274 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.153968 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.155446 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.156280 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.157006 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.157164 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.159617 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.159624 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.159642 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.159673 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.161701 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.162271 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.162278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.162284 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.162431 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.162746 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.162749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.162751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.163303 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.163504 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.164356 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.164665 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":127,"threads_started":1}
W20260811 02:57:30.167253 12537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:30.167325 12537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:30.167395 12535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:30.168146 12535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:30.173172 12535 catalog_manager.cc:1347] Generated new cluster ID: 584bbbfbbf7a4a54b91156efa8009ce0
I20260811 02:57:30.173240 12535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:30.185034 12535 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:30.185985 12535 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:30.194479 12535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Generated new TSK 0
I20260811 02:57:30.195163 12535 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:30.223798 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46701
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:30.332268 12547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:30.332494 12547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:30.332543 12547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:30.336539 12547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:30.336586 12547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:30.336632 12547 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:57:30.336659 12547 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:57:30.336707 12547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:30.341668 12547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:46701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:30.342555 12547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:30.343477 12547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:30.352383 12565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.352402 12558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.352416 12547 server_base.cc:1034] running on GCE node
W20260811 02:57:30.352384 12568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.352957 12547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:30.353559 12547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:30.354728 12547 hybrid_clock.cc:648] HybridClock initialized: now 1786417050354714 us; error 24 us; skew 500 ppm
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:30.357334 12547 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:30.358131 12547 webserver.cc:460] Webserver started at http://127.8.203.65:43915/ using document root <none> and password file <none>
I20260811 02:57:30.358631 12547 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:30.358706 12547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:30.358908 12547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:30.360836 12547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "ffe2d38d01e64d7f98c969a1f3875f1c"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "d6f2ca03cf4b4746e63036a4be6d17c5"
server_key_iv: "94463cf7e4c32d09fe45b0db91a734dd"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.361397 12547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "ffe2d38d01e64d7f98c969a1f3875f1c"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "d6f2ca03cf4b4746e63036a4be6d17c5"
server_key_iv: "94463cf7e4c32d09fe45b0db91a734dd"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.365056 12547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:57:30.367457 12577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.368568 12547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:30.368681 12547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "ffe2d38d01e64d7f98c969a1f3875f1c"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "d6f2ca03cf4b4746e63036a4be6d17c5"
server_key_iv: "94463cf7e4c32d09fe45b0db91a734dd"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.368780 12547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:30.382876 12547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:30.385913 12547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:30.386090 12547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:30.387456 12547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:30.387519 12547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.387576 12547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:30.387609 12547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:30.398677 12547 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:45845
I20260811 02:57:30.399574 12547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:30.400393 12811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:45845 every 8 connection(s)
I20260811 02:57:30.401278 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.388152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:33525 (local address 127.8.203.126:46701)
0811 02:57:30.388256 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.388259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.389812 (+ 1553us) server_negotiation.cc:407] Connection header received
0811 02:57:30.390873 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.390877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.390938 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.391014 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.392112 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.392637 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.393850 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.394097 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.394188 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.397707 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.397711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.397714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.397745 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.399195 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.399744 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.399748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.399749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.399828 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.400199 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.400203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.400205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.400437 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.400532 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.400985 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.401116 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 02:57:30.401984 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.389165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46701 (local address 127.8.203.65:33525)
0811 02:57:30.389658 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.389710 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.390667 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.391186 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.391193 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.391483 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:30.391972 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.391987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.392752 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.392755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.393731 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.393741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.394314 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.394321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.394457 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.394967 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:30.394990 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:30.397549 (+ 2559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:30.399303 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.399309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.399323 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.399623 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.399918 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.399921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.399923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.400090 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.400524 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:30.400530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:30.400755 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.400913 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.401222 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":168,"threads_started":1}
I20260811 02:57:30.403128 12812 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46701
I20260811 02:57:30.403264 12812 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:30.403725 12812 heartbeater.cc:511] Master 127.8.203.126:46701 requested a full tablet report, sending...
I20260811 02:57:30.405165 12405 ts_manager.cc:194] Registered new tserver with Master: ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845)
I20260811 02:57:30.406435 12405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:33525
I20260811 02:57:30.409806 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 12547
I20260811 02:57:30.409905 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:30.410156 9005 external_mini_cluster.cc:1428] Setting key fcd8e029e5616d6ccc1a1c8e94473def
I20260811 02:57:30.411379 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.405051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:48385 (local address 127.8.203.126:46701)
0811 02:57:30.405158 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.405162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.405328 (+ 166us) server_negotiation.cc:407] Connection header received
0811 02:57:30.405450 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.405452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.405488 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.405547 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.405978 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.406446 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.407359 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.407560 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.407632 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.408678 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.408681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.408683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.408703 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.409957 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.410346 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.410349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.410351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.410430 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.410672 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.410675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.410677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.410830 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.410918 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.411146 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.411239 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:57:30.411404 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.404943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46701 (local address 127.8.203.65:48385)
0811 02:57:30.405224 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.405238 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.405368 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.405577 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.405580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.405771 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:30.405885 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.405893 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.406563 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.406566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.407246 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.407258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.407663 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.407667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.407752 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.408144 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:30.408158 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:30.408559 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:30.410048 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.410052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.410054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.410261 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.410510 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.410513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.410515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.410594 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.410914 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:30.410917 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:30.411003 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.411084 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.411212 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:30.433588 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46701
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:30.543090 12828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:30.543334 12828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:30.543376 12828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:30.547217 12828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:30.547260 12828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:30.547297 12828 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:57:30.547313 12828 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:57:30.547343 12828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:30.552337 12828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:46701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:30.553260 12828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:30.554253 12828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:30.563482 12839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.563413 12840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.563617 12828 server_base.cc:1034] running on GCE node
W20260811 02:57:30.563402 12842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.564093 12828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:30.564692 12828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:30.565833 12828 hybrid_clock.cc:648] HybridClock initialized: now 1786417050565814 us; error 29 us; skew 500 ppm
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:30.568322 12828 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:30.569171 12828 webserver.cc:460] Webserver started at http://127.8.203.66:39227/ using document root <none> and password file <none>
I20260811 02:57:30.569736 12828 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:30.569803 12828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:30.570013 12828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:30.571949 12828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "b79939ab57af4ab68076e57f67865468"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "7a1a7c6536f06c23d094fbc928b3f2c9"
server_key_iv: "7debba5bfe27865747178d4a998e4b56"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.572512 12828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "b79939ab57af4ab68076e57f67865468"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "7a1a7c6536f06c23d094fbc928b3f2c9"
server_key_iv: "7debba5bfe27865747178d4a998e4b56"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.576082 12828 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 02:57:30.578423 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.579571 12828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:30.579677 12828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "b79939ab57af4ab68076e57f67865468"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "7a1a7c6536f06c23d094fbc928b3f2c9"
server_key_iv: "7debba5bfe27865747178d4a998e4b56"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.579767 12828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:30.598883 12828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:30.601624 12828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:30.601795 12828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:30.603241 12828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:30.603303 12828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.603366 12828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:30.603394 12828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:30.613365 12828 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:41873
I20260811 02:57:30.613564 13044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:41873 every 8 connection(s)
I20260811 02:57:30.614329 12828 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:30.616533 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.603952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:49967 (local address 127.8.203.126:46701)
0811 02:57:30.604101 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.604106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.604980 (+ 874us) server_negotiation.cc:407] Connection header received
0811 02:57:30.606052 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.606056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.606113 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.606191 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.607800 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.608394 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.610145 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.610437 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.610539 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.612877 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.612882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.612884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.612913 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.614337 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.615020 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.615023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.615025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.615129 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.615454 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.615458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.615460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.615651 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.615758 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.616242 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.616368 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:57:30.617247 12869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.604386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46701 (local address 127.8.203.66:49967)
0811 02:57:30.604817 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.604868 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.605789 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.606447 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.606457 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.606942 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:30.607620 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.607643 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.608517 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.608520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.610010 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.610024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.610597 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.610602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.610779 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.611282 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:30.611298 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:30.612710 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:30.614451 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.614457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.614471 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.614893 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.615210 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.615214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.615216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.615361 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.615759 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:30.615766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:30.616012 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.616175 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.616489 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":94,"threads_started":1}
I20260811 02:57:30.618407 13048 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46701
I20260811 02:57:30.618541 13048 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:30.619074 13048 heartbeater.cc:511] Master 127.8.203.126:46701 requested a full tablet report, sending...
I20260811 02:57:30.620100 12405 ts_manager.cc:194] Registered new tserver with Master: b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873)
I20260811 02:57:30.620609 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 12828
I20260811 02:57:30.620724 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:30.620781 12405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:49967
I20260811 02:57:30.621037 9005 external_mini_cluster.cc:1428] Setting key 5030564f1cda4609fabed1e30299d8e3
I20260811 02:57:30.627337 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.620675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:43077 (local address 127.8.203.126:46701)
0811 02:57:30.620782 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.620786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.621027 (+ 241us) server_negotiation.cc:407] Connection header received
0811 02:57:30.621210 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.621214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.621262 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.621367 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.621940 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.622570 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.623338 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.623615 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.623715 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.624647 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.624650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.624653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.624678 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.625867 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.626240 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.626243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.626245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.626318 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.626564 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.626567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.626569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.626767 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.626863 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.627096 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.627193 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:57:30.627357 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.620600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46701 (local address 127.8.203.66:43077)
0811 02:57:30.620950 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.620964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.621096 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.621484 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.621487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.621687 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:30.621810 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.621818 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.622699 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.622701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.623231 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.623237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.623712 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.623715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.623776 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.624198 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:30.624212 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:30.624519 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:30.625965 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.625968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.625970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.626153 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.626400 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.626403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.626405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.626485 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.626861 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:30.626864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:30.626951 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.627034 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.627165 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:30.645530 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46701
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:33411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:30.754494 13063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:30.754732 13063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:30.754777 13063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:30.758678 13063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:30.758731 13063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:30.758769 13063 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:57:30.758785 13063 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:57:30.758816 13063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:30.764026 13063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:33411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:46701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:30.764935 13063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:30.765888 13063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:30.775084 13070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.775092 13076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:30.775092 13072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:30.775403 13063 server_base.cc:1034] running on GCE node
I20260811 02:57:30.775767 13063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:30.776387 13063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:30.777534 13063 hybrid_clock.cc:648] HybridClock initialized: now 1786417050777517 us; error 27 us; skew 500 ppm
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:30.779975 13063 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:30.780797 13063 webserver.cc:460] Webserver started at http://127.8.203.67:33439/ using document root <none> and password file <none>
I20260811 02:57:30.781352 13063 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:30.781421 13063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:30.781616 13063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:30.783552 13063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "ca5c4446ed784f8caa74ce58d77e5119"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "928b54e36ede5ae18f2603450365e165"
server_key_iv: "73cb74676f8e6a553bc489256cd5b900"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.784072 13063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "ca5c4446ed784f8caa74ce58d77e5119"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "928b54e36ede5ae18f2603450365e165"
server_key_iv: "73cb74676f8e6a553bc489256cd5b900"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.787563 13063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 02:57:30.789829 13093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.790932 13063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:30.791035 13063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "ca5c4446ed784f8caa74ce58d77e5119"
format_stamp: "Formatted at 2026-08-11 02:57:30 on dist-test-slave-gjw7"
server_key: "928b54e36ede5ae18f2603450365e165"
server_key_iv: "73cb74676f8e6a553bc489256cd5b900"
server_key_version: "encryptionkey@0"
I20260811 02:57:30.791122 13063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:30.803453 13063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:30.806028 13063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:30.806193 13063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:30.807578 13063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:30.807626 13063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:30.807678 13063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:30.807708 13063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:30.817672 13063 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:41581
I20260811 02:57:30.817759 13305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:41581 every 8 connection(s)
I20260811 02:57:30.818562 13063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:30.821036 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 13063
I20260811 02:57:30.821132 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:30.821425 9005 external_mini_cluster.cc:1428] Setting key b8a17ec944f470cba50c296f294fcb4f
I20260811 02:57:30.821744 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.808363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:45657 (local address 127.8.203.126:46701)
0811 02:57:30.808474 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.808478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.809520 (+ 1042us) server_negotiation.cc:407] Connection header received
0811 02:57:30.810525 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.810530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.810594 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.810651 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.812061 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.812870 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.814128 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.814384 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.814486 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.818055 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.818062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.818065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.818092 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.819620 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.820191 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.820195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.820197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.820282 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.820613 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.820616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.820619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.820803 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.820896 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.821456 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.821582 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 02:57:30.822645 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.808890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46701 (local address 127.8.203.67:45657)
0811 02:57:30.809385 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.809428 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.810270 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.811062 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.811069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.811391 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:30.811901 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.811921 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.813008 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.813012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.814002 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.814013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.814549 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.814554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.814753 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.815966 (+ 1213us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:30.815988 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:30.817864 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:30.819745 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.819751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.819766 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.820086 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.820370 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.820373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.820375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.820515 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.820909 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:30.820916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:30.821179 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.821389 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.821705 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":121,"threads_started":1}
I20260811 02:57:30.823779 13306 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46701
I20260811 02:57:30.823918 13306 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:30.824452 13306 heartbeater.cc:511] Master 127.8.203.126:46701 requested a full tablet report, sending...
I20260811 02:57:30.825508 12404 ts_manager.cc:194] Registered new tserver with Master: ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:30.826234 12404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:45657
I20260811 02:57:30.831912 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.825993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46701 (local address 127.8.203.67:33471)
0811 02:57:30.826291 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.826305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.826396 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.826630 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.826633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.826839 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:30.826937 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.826943 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.827906 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.827909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.828448 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.828455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.828830 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.828832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.828892 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.829263 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:30.829273 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:30.829590 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:30.830866 (+ 1276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.830869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.830870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.831042 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.831234 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:30.831236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:30.831238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:30.831300 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:30.831562 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:30.831564 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:30.831629 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.831698 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.831792 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260811 02:57:30.831924 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.826061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:33471 (local address 127.8.203.126:46701)
0811 02:57:30.826159 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.826163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.826360 (+ 197us) server_negotiation.cc:407] Connection header received
0811 02:57:30.826477 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.826480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.826526 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.826598 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.827048 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.827784 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.828542 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.828748 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.828823 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.829713 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.829716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.829718 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.829739 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.830781 (+ 1042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.831110 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.831112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.831114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.831176 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.831362 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.831364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.831365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.831502 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.831570 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.831737 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.831820 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 02:57:30.835016 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:57:30 dist-test-slave-gjw7 krb5kdc[12331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:30.859100 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.852256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42020 (local address 127.8.203.126:46701)
0811 02:57:30.852381 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.852385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.852395 (+ 10us) server_negotiation.cc:407] Connection header received
0811 02:57:30.852430 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.852432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.852468 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.852525 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:30.853154 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.853682 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.854425 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.854624 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.854691 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.856745 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:30.856747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:30.856750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:30.856769 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:30.857890 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.858236 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.858238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.858239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.858309 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:30.858510 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:30.858512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:30.858513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:30.858660 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:30.858751 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.858900 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.858987 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:57:30.861662 12404 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:30.864737 12404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:30.878592 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.872488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45845 (local address 127.0.0.1:49200)
0811 02:57:30.872990 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.873040 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.873166 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.874755 (+ 1589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.874758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.874778 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:30.874872 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.874878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.876010 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.876014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.877048 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.877058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.878051 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.878055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.878169 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.878210 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.878317 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.878445 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":36,"thread_start_us":77,"threads_started":1}
I20260811 02:57:30.878651 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.873662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.8.203.65:45845)
0811 02:57:30.874347 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.874361 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.874380 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:57:30.874443 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.874448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.874629 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.874774 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:30.874978 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.875865 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.877178 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.877935 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.878036 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.878207 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.878357 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.878450 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":75,"threads_started":1}
I20260811 02:57:30.879027 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.872845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:41581 (local address 127.0.0.1:52796)
0811 02:57:30.874161 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.874174 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.874261 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.874923 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.874926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.874944 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:30.875030 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.875037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.876240 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.876244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.877386 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.877396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.878653 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.878656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.878741 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.878761 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.878828 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.878900 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"thread_start_us":104,"threads_started":1}
I20260811 02:57:30.879063 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.873632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.8.203.67:41581)
0811 02:57:30.873935 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.873944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.874349 (+ 405us) server_negotiation.cc:407] Connection header received
0811 02:57:30.874444 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.874448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.874629 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.874774 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:30.875154 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.876129 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.877518 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.878353 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.878448 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.878594 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.878879 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.878957 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":103,"threads_started":1}
I20260811 02:57:30.879346 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.872629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:41873 (local address 127.0.0.1:47910)
0811 02:57:30.873172 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:30.873183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:30.873251 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:30.874070 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:30.874076 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:30.874103 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:30.874235 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.874243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.875738 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.875741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.877078 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:30.877089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.878393 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.878397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:30.878600 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.878626 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:57:30.879187 (+ 561us) client_negotiation.cc:240] Negotiation successful
0811 02:57:30.879258 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":20,"thread_start_us":96,"threads_started":1}
I20260811 02:57:30.879665 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:30.873182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.8.203.66:41873)
0811 02:57:30.873543 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:30.873555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:30.873580 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:57:30.873666 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:30.873671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:30.873900 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:30.874066 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:30.874370 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.875613 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.877215 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:30.878042 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:30.878123 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:30.878279 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:30.879484 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0811 02:57:30.879563 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":139,"threads_started":1}
I20260811 02:57:30.881909 13186 tablet_service.cc:1467] Processing CreateTablet for tablet ea443a2956e0418d83ae7b22dbc19711 (DEFAULT_TABLE table=test-table [id=76cff63dfa5d4c1c9798a4d58d604990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:30.882247 12933 tablet_service.cc:1467] Processing CreateTablet for tablet ea443a2956e0418d83ae7b22dbc19711 (DEFAULT_TABLE table=test-table [id=76cff63dfa5d4c1c9798a4d58d604990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:30.883111 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea443a2956e0418d83ae7b22dbc19711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:30.883174 12933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea443a2956e0418d83ae7b22dbc19711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:30.883172 12686 tablet_service.cc:1467] Processing CreateTablet for tablet ea443a2956e0418d83ae7b22dbc19711 (DEFAULT_TABLE table=test-table [id=76cff63dfa5d4c1c9798a4d58d604990]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:30.884166 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea443a2956e0418d83ae7b22dbc19711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:30.889412 13346 tablet_bootstrap.cc:492] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Bootstrap starting.
I20260811 02:57:30.891034 13353 tablet_bootstrap.cc:492] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Bootstrap starting.
I20260811 02:57:30.891434 13350 tablet_bootstrap.cc:492] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Bootstrap starting.
I20260811 02:57:30.892334 13346 tablet_bootstrap.cc:654] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:30.893117 13353 tablet_bootstrap.cc:654] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:30.893189 13350 tablet_bootstrap.cc:654] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:30.893204 13346 log.cc:824] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:30.893908 13353 log.cc:824] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:30.893941 13350 log.cc:824] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:30.895534 13346 tablet_bootstrap.cc:492] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: No bootstrap required, opened a new log
I20260811 02:57:30.895727 13346 ts_tablet_manager.cc:1397] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:57:30.895761 13353 tablet_bootstrap.cc:492] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: No bootstrap required, opened a new log
I20260811 02:57:30.895913 13353 ts_tablet_manager.cc:1397] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:57:30.898437 13350 tablet_bootstrap.cc:492] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: No bootstrap required, opened a new log
I20260811 02:57:30.898608 13350 ts_tablet_manager.cc:1397] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260811 02:57:30.899621 13353 raft_consensus.cc:348] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:30.899621 13346 raft_consensus.cc:348] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:30.899894 13346 raft_consensus.cc:374] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:30.899894 13353 raft_consensus.cc:374] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:30.899952 13346 raft_consensus.cc:729] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79939ab57af4ab68076e57f67865468, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.899952 13353 raft_consensus.cc:729] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe2d38d01e64d7f98c969a1f3875f1c, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.900400 13346 consensus_queue.cc:260] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:30.900400 13353 consensus_queue.cc:260] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:30.901091 13048 heartbeater.cc:503] Master 127.8.203.126:46701 was elected leader, sending a full tablet report...
I20260811 02:57:30.901484 13346 ts_tablet_manager.cc:1428] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:57:30.901484 13353 ts_tablet_manager.cc:1428] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 02:57:30.901778 13350 raft_consensus.cc:348] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:30.902036 13350 raft_consensus.cc:374] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:30.902096 13350 raft_consensus.cc:729] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5c4446ed784f8caa74ce58d77e5119, State: Initialized, Role: FOLLOWER
I20260811 02:57:30.902518 12812 heartbeater.cc:503] Master 127.8.203.126:46701 was elected leader, sending a full tablet report...
I20260811 02:57:30.902670 13350 consensus_queue.cc:260] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:30.903290 13306 heartbeater.cc:503] Master 127.8.203.126:46701 was elected leader, sending a full tablet report...
I20260811 02:57:30.903611 13350 ts_tablet_manager.cc:1428] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:31.013903 13359 raft_consensus.cc:482] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:31.014113 13359 raft_consensus.cc:504] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.015288 13359 leader_election.cc:288] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:31.019269 12869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.015543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:41581 (local address 127.8.203.66:42141)
0811 02:57:31.015668 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.015683 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.015776 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.016022 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.016025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.016046 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:31.016163 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.016169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.017039 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.017042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.018021 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.018033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.018913 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.018916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.018992 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.019014 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.019072 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.019140 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 02:57:31.019300 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.015668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:42141 (local address 127.8.203.67:41581)
0811 02:57:31.015791 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.015794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.015808 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:31.015860 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.015863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.015905 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.015981 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:31.016273 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.016937 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.018150 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.018796 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.018921 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.018972 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.019108 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.019177 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:57:31.019976 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea443a2956e0418d83ae7b22dbc19711" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119" is_pre_election: true
I20260811 02:57:31.020277 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.015593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:58793 (local address 127.8.203.65:45845)
0811 02:57:31.015717 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.015722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.016265 (+ 543us) server_negotiation.cc:407] Connection header received
0811 02:57:31.016355 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.016357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.016406 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.016460 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:31.016734 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.017358 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.019252 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.019779 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.019869 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.019906 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.020112 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.020177 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:57:31.020306 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.015543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45845 (local address 127.8.203.66:58793)
0811 02:57:31.016172 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.016183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.016278 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.016489 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.016492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.016510 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:31.016630 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.016637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.017472 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.017474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.019118 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.019129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.019905 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.019908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.019998 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.020022 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.020103 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.020166 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":287,"thread_start_us":150,"threads_started":1}
I20260811 02:57:31.020303 13227 raft_consensus.cc:2454] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 0.
I20260811 02:57:31.020767 12857 leader_election.cc:302] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79939ab57af4ab68076e57f67865468, ca5c4446ed784f8caa74ce58d77e5119; no voters:
I20260811 02:57:31.020948 12717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea443a2956e0418d83ae7b22dbc19711" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" is_pre_election: true
I20260811 02:57:31.021034 13359 raft_consensus.cc:2781] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:31.021118 13359 raft_consensus.cc:482] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:31.021166 13359 raft_consensus.cc:3037] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.021212 12717 raft_consensus.cc:2454] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 0.
I20260811 02:57:31.022320 13359 raft_consensus.cc:504] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.022652 13359 leader_election.cc:288] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 election: Requested vote from peers ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:31.022917 12717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea443a2956e0418d83ae7b22dbc19711" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c"
I20260811 02:57:31.023016 12717 raft_consensus.cc:3037] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.023994 12717 raft_consensus.cc:2454] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 1.
I20260811 02:57:31.024261 12856 leader_election.cc:302] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79939ab57af4ab68076e57f67865468, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:31.024241 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea443a2956e0418d83ae7b22dbc19711" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119"
I20260811 02:57:31.024358 13227 raft_consensus.cc:3037] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.024420 13359 raft_consensus.cc:2781] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:31.024624 13359 raft_consensus.cc:686] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 1 LEADER]: Becoming Leader. State: Replica: b79939ab57af4ab68076e57f67865468, State: Running, Role: LEADER
I20260811 02:57:31.024951 13359 consensus_queue.cc:237] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.025547 13227 raft_consensus.cc:2454] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 1.
I20260811 02:57:31.028190 12405 catalog_manager.cc:5495] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 reported cstate change: term changed from 0 to 1, leader changed from <none> to b79939ab57af4ab68076e57f67865468 (127.8.203.66). New cstate: current_term: 1 leader_uuid: "b79939ab57af4ab68076e57f67865468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } health_report { overall_health: UNKNOWN } } }
W20260811 02:57:31.070422 13307 tablet.cc:2369] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 02:57:31 dist-test-slave-gjw7 krb5kdc[12331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417050, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:31.089972 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.082735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46701 (local address 127.0.0.1:42024)
0811 02:57:31.083039 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.083052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.083143 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.083340 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.083342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.083704 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:31.083796 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.083802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.084523 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.084525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.085071 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.085079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.085453 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.085455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.085516 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.085983 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:31.085995 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:31.087305 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:31.088680 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:31.088689 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:31.088694 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:31.088914 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:31.089107 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:31.089109 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:31.089110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:31.089180 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:31.089504 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:31.089509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:31.089609 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.089695 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.089811 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
I20260811 02:57:31.089991 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.082791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.8.203.126:46701)
0811 02:57:31.082919 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.082923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.083110 (+ 187us) server_negotiation.cc:407] Connection header received
0811 02:57:31.083205 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.083207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.083247 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.083311 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:31.083888 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.084433 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.085148 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.085376 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.085446 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.087442 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:31.087445 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:31.087447 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:31.087468 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:31.088585 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:31.088984 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:31.088986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:31.088987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:31.089050 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:31.089237 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:31.089239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:31.089240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:31.089421 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:31.089490 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.089732 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.089865 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":38}
I20260811 02:57:31.093437 12405 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:42024:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:57:31.094197 12405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:31.098766 12686 tablet_service.cc:1467] Processing CreateTablet for tablet af84270810b24aeba905117c1ea62d3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d8fa21cde5a4446e833359669351d2b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:57:31.098903 13186 tablet_service.cc:1467] Processing CreateTablet for tablet af84270810b24aeba905117c1ea62d3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d8fa21cde5a4446e833359669351d2b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:57:31.098907 12933 tablet_service.cc:1467] Processing CreateTablet for tablet af84270810b24aeba905117c1ea62d3c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d8fa21cde5a4446e833359669351d2b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:57:31.099447 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af84270810b24aeba905117c1ea62d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.099560 12933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af84270810b24aeba905117c1ea62d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.099560 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af84270810b24aeba905117c1ea62d3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.101917 13353 tablet_bootstrap.cc:492] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: Bootstrap starting.
I20260811 02:57:31.102375 13346 tablet_bootstrap.cc:492] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: Bootstrap starting.
I20260811 02:57:31.102605 13350 tablet_bootstrap.cc:492] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: Bootstrap starting.
I20260811 02:57:31.102897 13353 tablet_bootstrap.cc:654] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.103247 13346 tablet_bootstrap.cc:654] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.103569 13350 tablet_bootstrap.cc:654] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.104193 13353 tablet_bootstrap.cc:492] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: No bootstrap required, opened a new log
I20260811 02:57:31.104282 13353 ts_tablet_manager.cc:1397] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:31.104499 13346 tablet_bootstrap.cc:492] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: No bootstrap required, opened a new log
I20260811 02:57:31.104573 13346 ts_tablet_manager.cc:1397] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:31.104869 13350 tablet_bootstrap.cc:492] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: No bootstrap required, opened a new log
I20260811 02:57:31.104947 13350 ts_tablet_manager.cc:1397] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:31.104883 13353 raft_consensus.cc:348] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.104988 13353 raft_consensus.cc:374] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.105019 13353 raft_consensus.cc:729] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe2d38d01e64d7f98c969a1f3875f1c, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.104979 13346 raft_consensus.cc:348] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.105057 13346 raft_consensus.cc:374] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.105077 13346 raft_consensus.cc:729] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79939ab57af4ab68076e57f67865468, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.105142 13353 consensus_queue.cc:260] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.105180 13346 consensus_queue.cc:260] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.105403 13350 raft_consensus.cc:348] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.105460 13346 ts_tablet_manager.cc:1428] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:31.105502 13350 raft_consensus.cc:374] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.105435 13368 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.105531 13350 raft_consensus.cc:729] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5c4446ed784f8caa74ce58d77e5119, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.105572 13368 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:31.105638 13350 consensus_queue.cc:260] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.105733 13359 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.105839 13359 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:31.105882 13350 ts_tablet_manager.cc:1428] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:31.105906 13369 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.106030 13369 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:31.106518 13353 ts_tablet_manager.cc:1428] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260811 02:57:31.115414 13050 tablet.cc:2369] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:31.150858 12815 tablet.cc:2369] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:31.185012 13369 raft_consensus.cc:482] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:31.185135 13369 raft_consensus.cc:504] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.186249 13369 leader_election.cc:288] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845), b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873)
I20260811 02:57:31.189600 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.186560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56329 (local address 127.8.203.65:45845)
0811 02:57:31.186692 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.186696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.186712 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:57:31.186833 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.186836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.186880 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.186941 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:31.187199 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.187697 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.188693 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.189074 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.189174 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.189229 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.189425 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.189497 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:57:31.189600 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.186441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45845 (local address 127.8.203.67:56329)
0811 02:57:31.186623 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.186638 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.186727 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.186964 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.186967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.186986 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:31.187096 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.187101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.187798 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.187801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.188570 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.188579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.189174 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.189178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.189282 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.189327 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.189400 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.189471 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 02:57:31.189920 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.186704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:47337 (local address 127.8.203.66:41873)
0811 02:57:31.186837 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.186841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.186986 (+ 145us) server_negotiation.cc:407] Connection header received
0811 02:57:31.187108 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.187111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.187151 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.187213 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:31.187495 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.187976 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.188972 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.189373 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.189476 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.189528 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.189732 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.189808 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:57:31.190052 12717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af84270810b24aeba905117c1ea62d3c" candidate_uuid: "ca5c4446ed784f8caa74ce58d77e5119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" is_pre_election: true
I20260811 02:57:31.190205 12717 raft_consensus.cc:2454] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5c4446ed784f8caa74ce58d77e5119 in term 0.
I20260811 02:57:31.190253 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.186499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:41873 (local address 127.8.203.67:47337)
0811 02:57:31.186901 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.186912 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.187008 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.187235 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.187238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.187253 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:31.187387 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.187393 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.188079 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.188082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.188851 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.188860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.189493 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.189497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.189594 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.189614 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.190057 (+ 443us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.190126 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":110,"threads_started":1}
I20260811 02:57:31.190604 13094 leader_election.cc:302] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca5c4446ed784f8caa74ce58d77e5119, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:31.190754 12975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af84270810b24aeba905117c1ea62d3c" candidate_uuid: "ca5c4446ed784f8caa74ce58d77e5119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79939ab57af4ab68076e57f67865468" is_pre_election: true
I20260811 02:57:31.190860 13369 raft_consensus.cc:2781] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:31.190927 13369 raft_consensus.cc:482] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:31.190968 13369 raft_consensus.cc:3037] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.190982 12975 raft_consensus.cc:2454] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5c4446ed784f8caa74ce58d77e5119 in term 0.
I20260811 02:57:31.192040 13369 raft_consensus.cc:504] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.192389 13369 leader_election.cc:288] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [CANDIDATE]: Term 1 election: Requested vote from peers ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845), b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873)
I20260811 02:57:31.192639 12717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af84270810b24aeba905117c1ea62d3c" candidate_uuid: "ca5c4446ed784f8caa74ce58d77e5119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c"
I20260811 02:57:31.192711 12975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af84270810b24aeba905117c1ea62d3c" candidate_uuid: "ca5c4446ed784f8caa74ce58d77e5119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79939ab57af4ab68076e57f67865468"
I20260811 02:57:31.192770 12717 raft_consensus.cc:3037] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.192812 12975 raft_consensus.cc:3037] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.193938 12717 raft_consensus.cc:2454] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5c4446ed784f8caa74ce58d77e5119 in term 1.
I20260811 02:57:31.193938 12975 raft_consensus.cc:2454] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5c4446ed784f8caa74ce58d77e5119 in term 1.
I20260811 02:57:31.194202 13094 leader_election.cc:302] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca5c4446ed784f8caa74ce58d77e5119, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:31.194350 13369 raft_consensus.cc:2781] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:31.194536 13369 raft_consensus.cc:686] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [term 1 LEADER]: Becoming Leader. State: Replica: ca5c4446ed784f8caa74ce58d77e5119, State: Running, Role: LEADER
I20260811 02:57:31.194881 13369 consensus_queue.cc:237] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.197131 13396 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: TxnStatusTablet state changed. Reason: New leader ca5c4446ed784f8caa74ce58d77e5119. Latest consensus state: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.197242 13396 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:57:31.197769 13401 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:57:31.197973 13401 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:57:31.198160 12406 catalog_manager.cc:5495] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67). New cstate: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:31.232086 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.228444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:41581 (local address 127.0.0.1:52810)
0811 02:57:31.228598 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.228614 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.228726 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.229049 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.229053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.229060 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:31.229179 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.229185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.229922 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.229926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.230547 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.230556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.230990 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.230994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.231093 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.231140 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:31.231786 (+ 646us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:31.231803 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.231868 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.231957 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 02:57:31.232100 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.228555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.8.203.67:41581)
0811 02:57:31.228698 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.228702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.228716 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:31.228884 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.228886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.228942 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.229025 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:31.229309 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.229811 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.230669 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.230887 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.230975 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.231285 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:31.231687 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:31.231800 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.231880 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.231930 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:57:31.236994 12975 raft_consensus.cc:1261] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Refusing update from remote peer ca5c4446ed784f8caa74ce58d77e5119: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:31.237099 12717 raft_consensus.cc:1261] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Refusing update from remote peer ca5c4446ed784f8caa74ce58d77e5119: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:31.237618 13369 consensus_queue.cc:1035] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:31.237797 13396 consensus_queue.cc:1035] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:31.240764 13359 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: TxnStatusTablet state changed. Reason: New leader ca5c4446ed784f8caa74ce58d77e5119. Latest consensus state: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.240883 13359 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:31.241251 13368 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: TxnStatusTablet state changed. Reason: New leader ca5c4446ed784f8caa74ce58d77e5119. Latest consensus state: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.241381 13368 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:31.243474 13396 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.243597 13396 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:57:31.244058 13405 mvcc.cc:205] Tried to move back new op lower bound from 7317164241859977216 to 7317164241696964608. Current Snapshot: MvccSnapshot[applied={T|T < 7317164241859977216}]
I20260811 02:57:31.245406 13368 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.245519 13368 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P ffe2d38d01e64d7f98c969a1f3875f1c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:31.246891 13369 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.247001 13369 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P ca5c4446ed784f8caa74ce58d77e5119: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:57:31.248107 13407 mvcc.cc:205] Tried to move back new op lower bound from 7317164241859977216 to 7317164241696964608. Current Snapshot: MvccSnapshot[applied={T|T < 7317164241859977216}]
I20260811 02:57:31.248107 13359 tablet_replica.cc:439] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca5c4446ed784f8caa74ce58d77e5119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } } }
I20260811 02:57:31.248338 13359 tablet_replica.cc:442] T af84270810b24aeba905117c1ea62d3c P b79939ab57af4ab68076e57f67865468: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:57:31.255607 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.252094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.8.203.66:41873)
0811 02:57:31.252196 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.252200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.252263 (+ 63us) server_negotiation.cc:407] Connection header received
0811 02:57:31.252433 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.252437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.252490 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.252572 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:31.252810 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.253358 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.254189 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.254428 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.254522 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.254864 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:31.255254 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:31.255354 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.255432 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.255483 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:57:31.266579 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.262948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:60335 (local address 127.8.203.67:41581)
0811 02:57:31.263071 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.263076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.263345 (+ 269us) server_negotiation.cc:407] Connection header received
0811 02:57:31.263474 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.263477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.263520 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.263577 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:31.263858 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.264388 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.265591 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.265807 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.265878 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.266220 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:31.266288 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:31.266335 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.266432 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.266477 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:57:31.266654 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.262866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:41581 (local address 127.8.203.66:60335)
0811 02:57:31.263252 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.263267 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.263389 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.263601 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.263604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.263612 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:31.263742 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.263748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.264485 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.264488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.265491 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.265503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.265917 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.265921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.266022 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.266079 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:31.266355 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:31.266360 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.266416 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.266496 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":107,"threads_started":1}
I20260811 02:57:31.273494 12717 raft_consensus.cc:1261] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Refusing update from remote peer b79939ab57af4ab68076e57f67865468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:31.273707 13227 raft_consensus.cc:1261] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Refusing update from remote peer b79939ab57af4ab68076e57f67865468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:31.274001 13377 consensus_queue.cc:1035] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:31.274155 13359 consensus_queue.cc:1035] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:31.279887 13408 mvcc.cc:205] Tried to move back new op lower bound from 7317164242011951104 to 7317164241001144320. Current Snapshot: MvccSnapshot[applied={T|T < 7317164242011951104}]
I20260811 02:57:31.296952 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.293280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:41873 (local address 127.8.203.67:37887)
0811 02:57:31.293526 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:31.293539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:31.293616 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:31.293831 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:31.293833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:31.293838 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:31.293928 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.293934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.294938 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.294942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.295784 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:31.295795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.296360 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.296362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:31.296423 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.296467 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:31.296743 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:31.296747 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:57:31.296789 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 02:57:31.296849 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 02:57:31.296979 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:31.293390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:37887 (local address 127.8.203.66:41873)
0811 02:57:31.293504 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:31.293509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:31.293588 (+ 79us) server_negotiation.cc:407] Connection header received
0811 02:57:31.293686 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:31.293689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:31.293747 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:31.293816 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:31.294026 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.294844 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.295894 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:31.296258 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:31.296363 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:31.296585 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:31.296665 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:31.296746 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:31.296803 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 02:57:31.296859 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:57:31.323472 12406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:31.323731 12406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:31.326514 12406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Sending DeleteTablet for 3 replicas of tablet ea443a2956e0418d83ae7b22dbc19711
I20260811 02:57:31.327253 12686 tablet_service.cc:1514] Processing DeleteTablet for tablet ea443a2956e0418d83ae7b22dbc19711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:31 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:49200
I20260811 02:57:31.327826 13437 tablet_replica.cc:330] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: stopping tablet replica
I20260811 02:57:31.327889 13186 tablet_service.cc:1514] Processing DeleteTablet for tablet ea443a2956e0418d83ae7b22dbc19711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:31 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52796
I20260811 02:57:31.328155 13437 raft_consensus.cc:2229] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:31.328387 13437 raft_consensus.cc:2258] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:31.329033 12406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:31.329178 12933 tablet_service.cc:1514] Processing DeleteTablet for tablet ea443a2956e0418d83ae7b22dbc19711 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:31 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:47910
W20260811 02:57:31.329461 12406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:31.329881 13438 tablet_replica.cc:330] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: stopping tablet replica
I20260811 02:57:31.331264 13437 ts_tablet_manager.cc:1905] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:31.334484 13437 ts_tablet_manager.cc:1918] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:31.335099 12933 tablet_service.cc:1467] Processing CreateTablet for tablet 33f606222fd44d6882e2b991e248899c (DEFAULT_TABLE table=test-table [id=cc9ef4c206a845b7a1a1f149d860b2a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:31.335376 12933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33f606222fd44d6882e2b991e248899c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.335727 13186 tablet_service.cc:1467] Processing CreateTablet for tablet 33f606222fd44d6882e2b991e248899c (DEFAULT_TABLE table=test-table [id=cc9ef4c206a845b7a1a1f149d860b2a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:31.335983 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33f606222fd44d6882e2b991e248899c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.337926 13346 tablet_bootstrap.cc:492] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: Bootstrap starting.
I20260811 02:57:31.338845 13346 tablet_bootstrap.cc:654] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.339124 13438 raft_consensus.cc:2229] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:31.339354 13438 raft_consensus.cc:2258] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:31.339453 13350 tablet_bootstrap.cc:492] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: Bootstrap starting.
I20260811 02:57:31.340031 13440 tablet_replica.cc:330] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: stopping tablet replica
I20260811 02:57:31.340324 13350 tablet_bootstrap.cc:654] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.340325 13440 raft_consensus.cc:2229] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:31.340680 13440 raft_consensus.cc:2258] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:31.334582 13437 log.cc:1196] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/ea443a2956e0418d83ae7b22dbc19711
I20260811 02:57:31.335460 12686 tablet_service.cc:1467] Processing CreateTablet for tablet 33f606222fd44d6882e2b991e248899c (DEFAULT_TABLE table=test-table [id=cc9ef4c206a845b7a1a1f149d860b2a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:31.341221 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33f606222fd44d6882e2b991e248899c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:31.341820 13438 ts_tablet_manager.cc:1905] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:31.341984 13440 ts_tablet_manager.cc:1905] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:31.343302 13353 tablet_bootstrap.cc:492] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: Bootstrap starting.
I20260811 02:57:31.344163 13353 tablet_bootstrap.cc:654] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:31.344635 13440 ts_tablet_manager.cc:1918] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:31.344750 13440 log.cc:1196] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/ea443a2956e0418d83ae7b22dbc19711
I20260811 02:57:31.345065 13440 ts_tablet_manager.cc:1939] T ea443a2956e0418d83ae7b22dbc19711 P b79939ab57af4ab68076e57f67865468: Deleting consensus metadata
I20260811 02:57:31.345194 13437 ts_tablet_manager.cc:1939] T ea443a2956e0418d83ae7b22dbc19711 P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting consensus metadata
I20260811 02:57:31.345935 13346 tablet_bootstrap.cc:492] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: No bootstrap required, opened a new log
I20260811 02:57:31.346021 13346 ts_tablet_manager.cc:1397] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260811 02:57:31.346259 12385 catalog_manager.cc:4841] TS b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873): tablet ea443a2956e0418d83ae7b22dbc19711 (table test-table [id=76cff63dfa5d4c1c9798a4d58d604990]) successfully deleted
I20260811 02:57:31.346462 13346 raft_consensus.cc:348] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.346580 13346 raft_consensus.cc:374] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.346621 13346 raft_consensus.cc:729] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79939ab57af4ab68076e57f67865468, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.346701 12381 catalog_manager.cc:4841] TS ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845): tablet ea443a2956e0418d83ae7b22dbc19711 (table test-table [id=76cff63dfa5d4c1c9798a4d58d604990]) successfully deleted
I20260811 02:57:31.346750 13346 consensus_queue.cc:260] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.347010 13346 ts_tablet_manager.cc:1428] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:31.347482 13353 tablet_bootstrap.cc:492] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: No bootstrap required, opened a new log
I20260811 02:57:31.347558 13353 ts_tablet_manager.cc:1397] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 02:57:31.347950 13353 raft_consensus.cc:348] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.348044 13353 raft_consensus.cc:374] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.348070 13353 raft_consensus.cc:729] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe2d38d01e64d7f98c969a1f3875f1c, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.348172 13353 consensus_queue.cc:260] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.348392 13353 ts_tablet_manager.cc:1428] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:31.350664 13438 ts_tablet_manager.cc:1918] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:31.350865 13350 tablet_bootstrap.cc:492] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: No bootstrap required, opened a new log
I20260811 02:57:31.350852 13438 log.cc:1196] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/ea443a2956e0418d83ae7b22dbc19711
I20260811 02:57:31.350976 13350 ts_tablet_manager.cc:1397] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260811 02:57:31.351467 13350 raft_consensus.cc:348] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.351570 13350 raft_consensus.cc:374] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:31.351599 13350 raft_consensus.cc:729] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5c4446ed784f8caa74ce58d77e5119, State: Initialized, Role: FOLLOWER
I20260811 02:57:31.351714 13350 consensus_queue.cc:260] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.351967 13350 ts_tablet_manager.cc:1428] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:57:31.352485 13438 ts_tablet_manager.cc:1939] T ea443a2956e0418d83ae7b22dbc19711 P ca5c4446ed784f8caa74ce58d77e5119: Deleting consensus metadata
I20260811 02:57:31.354955 12383 catalog_manager.cc:4841] TS ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581): tablet ea443a2956e0418d83ae7b22dbc19711 (table test-table [id=76cff63dfa5d4c1c9798a4d58d604990]) successfully deleted
I20260811 02:57:31.413878 13359 raft_consensus.cc:482] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:31.414016 13359 raft_consensus.cc:504] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.414420 13359 leader_election.cc:288] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:31.414815 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f606222fd44d6882e2b991e248899c" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" is_pre_election: true
I20260811 02:57:31.414906 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f606222fd44d6882e2b991e248899c" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119" is_pre_election: true
I20260811 02:57:31.414955 12716 raft_consensus.cc:2454] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 0.
I20260811 02:57:31.415050 13227 raft_consensus.cc:2454] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 0.
I20260811 02:57:31.415275 12856 leader_election.cc:302] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79939ab57af4ab68076e57f67865468, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:31.415460 13359 raft_consensus.cc:2781] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:31.415517 13359 raft_consensus.cc:482] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:31.415563 13359 raft_consensus.cc:3037] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.416479 13359 raft_consensus.cc:504] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.416813 13359 leader_election.cc:288] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 election: Requested vote from peers ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:31.417217 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f606222fd44d6882e2b991e248899c" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c"
I20260811 02:57:31.417213 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33f606222fd44d6882e2b991e248899c" candidate_uuid: "b79939ab57af4ab68076e57f67865468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119"
I20260811 02:57:31.417343 13227 raft_consensus.cc:3037] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.417352 12716 raft_consensus.cc:3037] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:31.418445 12716 raft_consensus.cc:2454] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 1.
I20260811 02:57:31.418504 13227 raft_consensus.cc:2454] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b79939ab57af4ab68076e57f67865468 in term 1.
I20260811 02:57:31.418725 12856 leader_election.cc:302] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79939ab57af4ab68076e57f67865468, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:31.419005 13359 raft_consensus.cc:2781] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:31.419081 13359 raft_consensus.cc:686] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 1 LEADER]: Becoming Leader. State: Replica: b79939ab57af4ab68076e57f67865468, State: Running, Role: LEADER
I20260811 02:57:31.419214 13359 consensus_queue.cc:237] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:31.420815 12406 catalog_manager.cc:5495] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 reported cstate change: term changed from 0 to 1, leader changed from <none> to b79939ab57af4ab68076e57f67865468 (127.8.203.66). New cstate: current_term: 1 leader_uuid: "b79939ab57af4ab68076e57f67865468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260811 02:57:31.451254 13227 raft_consensus.cc:1261] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Refusing update from remote peer b79939ab57af4ab68076e57f67865468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:31.451431 12716 raft_consensus.cc:1261] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Refusing update from remote peer b79939ab57af4ab68076e57f67865468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:31.451620 13359 consensus_queue.cc:1035] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:31.451862 13430 consensus_queue.cc:1035] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:57:31.468258 13407 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 02:57:33.017417 12404 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:33.017563 12404 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:33.019865 12404 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Sending DeleteTablet for 3 replicas of tablet 33f606222fd44d6882e2b991e248899c
I20260811 02:57:33.020332 12686 tablet_service.cc:1514] Processing DeleteTablet for tablet 33f606222fd44d6882e2b991e248899c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:33 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:49200
I20260811 02:57:33.020401 12933 tablet_service.cc:1514] Processing DeleteTablet for tablet 33f606222fd44d6882e2b991e248899c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:33 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:47910
I20260811 02:57:33.020694 13505 tablet_replica.cc:330] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: stopping tablet replica
I20260811 02:57:33.020776 13186 tablet_service.cc:1514] Processing DeleteTablet for tablet 33f606222fd44d6882e2b991e248899c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:33 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52796
I20260811 02:57:33.020828 13505 raft_consensus.cc:2229] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:33.021008 13505 raft_consensus.cc:2258] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:33.021257 13504 tablet_replica.cc:330] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: stopping tablet replica
I20260811 02:57:33.021092 13506 tablet_replica.cc:330] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: stopping tablet replica
I20260811 02:57:33.021409 13504 raft_consensus.cc:2229] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:33.021409 13506 raft_consensus.cc:2229] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:33.021513 13504 raft_consensus.cc:2258] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:33.021520 13506 raft_consensus.cc:2258] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:33.021829 13505 ts_tablet_manager.cc:1905] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:33.022109 13506 ts_tablet_manager.cc:1905] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:33.024482 13504 ts_tablet_manager.cc:1905] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:33.024592 13506 ts_tablet_manager.cc:1918] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:57:33.024605 13505 ts_tablet_manager.cc:1918] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:57:33.024675 13506 log.cc:1196] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/33f606222fd44d6882e2b991e248899c
I20260811 02:57:33.024714 13505 log.cc:1196] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/33f606222fd44d6882e2b991e248899c
I20260811 02:57:33.024973 13506 ts_tablet_manager.cc:1939] T 33f606222fd44d6882e2b991e248899c P ca5c4446ed784f8caa74ce58d77e5119: Deleting consensus metadata
I20260811 02:57:33.024981 13505 ts_tablet_manager.cc:1939] T 33f606222fd44d6882e2b991e248899c P b79939ab57af4ab68076e57f67865468: Deleting consensus metadata
I20260811 02:57:33.025590 12383 catalog_manager.cc:4841] TS ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581): tablet 33f606222fd44d6882e2b991e248899c (table test-table [id=cc9ef4c206a845b7a1a1f149d860b2a6]) successfully deleted
I20260811 02:57:33.025625 12385 catalog_manager.cc:4841] TS b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873): tablet 33f606222fd44d6882e2b991e248899c (table test-table [id=cc9ef4c206a845b7a1a1f149d860b2a6]) successfully deleted
I20260811 02:57:33.026923 13504 ts_tablet_manager.cc:1918] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:57:33.026991 13504 log.cc:1196] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/33f606222fd44d6882e2b991e248899c
I20260811 02:57:33.027180 13504 ts_tablet_manager.cc:1939] T 33f606222fd44d6882e2b991e248899c P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting consensus metadata
I20260811 02:57:33.027741 12381 catalog_manager.cc:4841] TS ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845): tablet 33f606222fd44d6882e2b991e248899c (table test-table [id=cc9ef4c206a845b7a1a1f149d860b2a6]) successfully deleted
Aug 11 02:57:33 dist-test-slave-gjw7 krb5kdc[12331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:57:33 dist-test-slave-gjw7 krb5kdc[12331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:33.055442 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.040447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.8.203.126:46701)
0811 02:57:33.040705 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:33.040711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:33.040725 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:33.040831 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:33.040833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:33.040878 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:33.040952 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:33.041657 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.042197 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.042948 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.043157 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.043221 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.052900 (+ 9679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:33.052903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:33.052906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:33.052929 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:33.054168 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:33.054543 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:33.054545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:33.054546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:33.054616 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:33.054793 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:33.054795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:33.054796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:33.054969 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:33.055067 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:33.055217 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 02:57:33.055313 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":99,"threads_started":1}
I20260811 02:57:33.057773 12404 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:33.058159 12404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:33.062620 12933 tablet_service.cc:1467] Processing CreateTablet for tablet 240546de0c3f4bddbdc881cc22a9cce5 (DEFAULT_TABLE table=test-table [id=3e4fc1d47ae649b09d75742384d62553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:33.062644 13186 tablet_service.cc:1467] Processing CreateTablet for tablet 240546de0c3f4bddbdc881cc22a9cce5 (DEFAULT_TABLE table=test-table [id=3e4fc1d47ae649b09d75742384d62553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:33.062644 12686 tablet_service.cc:1467] Processing CreateTablet for tablet 240546de0c3f4bddbdc881cc22a9cce5 (DEFAULT_TABLE table=test-table [id=3e4fc1d47ae649b09d75742384d62553]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:33.062914 12933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 240546de0c3f4bddbdc881cc22a9cce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:33.062928 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 240546de0c3f4bddbdc881cc22a9cce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:33.062928 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 240546de0c3f4bddbdc881cc22a9cce5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:33.065282 13524 tablet_bootstrap.cc:492] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: Bootstrap starting.
I20260811 02:57:33.065604 13523 tablet_bootstrap.cc:492] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: Bootstrap starting.
I20260811 02:57:33.065836 13525 tablet_bootstrap.cc:492] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: Bootstrap starting.
I20260811 02:57:33.066201 13524 tablet_bootstrap.cc:654] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:33.066386 13523 tablet_bootstrap.cc:654] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:33.066803 13525 tablet_bootstrap.cc:654] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:33.067420 13524 tablet_bootstrap.cc:492] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: No bootstrap required, opened a new log
I20260811 02:57:33.067518 13523 tablet_bootstrap.cc:492] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: No bootstrap required, opened a new log
I20260811 02:57:33.067541 13524 ts_tablet_manager.cc:1397] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:33.067611 13523 ts_tablet_manager.cc:1397] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:33.067981 13523 raft_consensus.cc:348] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.067987 13524 raft_consensus.cc:348] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.068076 13523 raft_consensus.cc:374] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:33.068089 13524 raft_consensus.cc:374] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:33.068099 13523 raft_consensus.cc:729] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79939ab57af4ab68076e57f67865468, State: Initialized, Role: FOLLOWER
I20260811 02:57:33.068133 13524 raft_consensus.cc:729] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5c4446ed784f8caa74ce58d77e5119, State: Initialized, Role: FOLLOWER
I20260811 02:57:33.068045 13525 tablet_bootstrap.cc:492] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: No bootstrap required, opened a new log
I20260811 02:57:33.068223 13523 consensus_queue.cc:260] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.068266 13525 ts_tablet_manager.cc:1397] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:33.068272 13524 consensus_queue.cc:260] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.068506 13523 ts_tablet_manager.cc:1428] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:57:33.068548 13524 ts_tablet_manager.cc:1428] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:33.069327 13525 raft_consensus.cc:348] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.069454 13525 raft_consensus.cc:374] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:33.069494 13525 raft_consensus.cc:729] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe2d38d01e64d7f98c969a1f3875f1c, State: Initialized, Role: FOLLOWER
I20260811 02:57:33.069617 13525 consensus_queue.cc:260] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.069954 13525 ts_tablet_manager.cc:1428] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:33.178316 13534 raft_consensus.cc:482] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:33.178455 13534 raft_consensus.cc:504] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.179566 13534 leader_election.cc:288] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:33.184669 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.179781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:41581 (local address 127.8.203.65:42943)
0811 02:57:33.180294 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:33.180305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:33.180396 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:33.180612 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:33.180614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:33.180651 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:33.180843 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.180851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.181900 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.181904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.183334 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.183348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.184216 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.184220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.184321 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.184350 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:57:33.184428 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 02:57:33.184507 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":131,"spinlock_wait_cycles":7552,"thread_start_us":120,"threads_started":1}
I20260811 02:57:33.184854 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.179781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:41873 (local address 127.8.203.65:50607)
0811 02:57:33.180184 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:33.180199 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:33.180328 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:33.180604 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:33.180607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:33.180664 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:33.180854 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.180861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.181919 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.181922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.183334 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.183348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.184352 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.184357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.184467 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.184494 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:57:33.184561 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:57:33.184633 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":99,"spinlock_wait_cycles":68864,"thread_start_us":142,"threads_started":1}
I20260811 02:57:33.184917 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.179884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:50607 (local address 127.8.203.66:41873)
0811 02:57:33.180182 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:33.180191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:33.180274 (+ 83us) server_negotiation.cc:407] Connection header received
0811 02:57:33.180431 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:33.180434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:33.180487 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:33.180588 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:33.180957 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.181801 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.183475 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.184143 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.184282 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.184329 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:33.184721 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 02:57:33.184801 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":110,"threads_started":1}
I20260811 02:57:33.185245 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.179875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:42943 (local address 127.8.203.67:41581)
0811 02:57:33.180182 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:33.180191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:33.180390 (+ 199us) server_negotiation.cc:407] Connection header received
0811 02:57:33.180473 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:33.180476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:33.180524 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:33.180596 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:33.180957 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.181801 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.183456 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.184091 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.184215 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.184275 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:33.184575 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 02:57:33.184647 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":118,"threads_started":1}
I20260811 02:57:33.185317 12975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240546de0c3f4bddbdc881cc22a9cce5" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79939ab57af4ab68076e57f67865468" is_pre_election: true
I20260811 02:57:33.185467 12975 raft_consensus.cc:2454] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 0.
I20260811 02:57:33.185607 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240546de0c3f4bddbdc881cc22a9cce5" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119" is_pre_election: true
I20260811 02:57:33.185747 13227 raft_consensus.cc:2454] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 0.
I20260811 02:57:33.185829 12586 leader_election.cc:302] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79939ab57af4ab68076e57f67865468, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:33.186100 13534 raft_consensus.cc:2781] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:33.186167 13534 raft_consensus.cc:482] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:33.186209 13534 raft_consensus.cc:3037] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:33.187172 13534 raft_consensus.cc:504] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.187461 13534 leader_election.cc:288] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 election: Requested vote from peers b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:33.187855 12975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240546de0c3f4bddbdc881cc22a9cce5" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79939ab57af4ab68076e57f67865468"
I20260811 02:57:33.187844 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "240546de0c3f4bddbdc881cc22a9cce5" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119"
I20260811 02:57:33.187961 13227 raft_consensus.cc:3037] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:33.187965 12975 raft_consensus.cc:3037] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:33.188803 12975 raft_consensus.cc:2454] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 1.
I20260811 02:57:33.188946 13227 raft_consensus.cc:2454] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 1.
I20260811 02:57:33.189042 12586 leader_election.cc:302] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79939ab57af4ab68076e57f67865468, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:33.189208 13534 raft_consensus.cc:2781] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:33.189447 13534 raft_consensus.cc:686] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 LEADER]: Becoming Leader. State: Replica: ffe2d38d01e64d7f98c969a1f3875f1c, State: Running, Role: LEADER
I20260811 02:57:33.189693 13534 consensus_queue.cc:237] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.192121 12404 catalog_manager.cc:5495] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65). New cstate: current_term: 1 leader_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260811 02:57:33.225750 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.222386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.8.203.65:45845)
0811 02:57:33.222635 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:33.222641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:33.222652 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:57:33.222724 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:33.222726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:33.222776 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:33.222886 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:33.223050 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.223597 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.224413 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.224677 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.224752 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.224975 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:33.225447 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:33.225529 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:33.225572 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 02:57:33.225617 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":107,"threads_started":1}
I20260811 02:57:33.234859 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.232013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:49353 (local address 127.8.203.67:41581)
0811 02:57:33.232113 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:33.232117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:33.232335 (+ 218us) server_negotiation.cc:407] Connection header received
0811 02:57:33.232433 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:33.232435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:33.232477 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:33.232532 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:33.232697 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.233158 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.234077 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.234267 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.234331 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.234570 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:33.234624 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:33.234668 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:33.234729 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 02:57:33.234766 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 02:57:33.234892 13551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.231924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:41581 (local address 127.8.203.65:49353)
0811 02:57:33.232267 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:33.232279 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:33.232363 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:33.232543 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:33.232545 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:33.232551 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:33.232630 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.232634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.233229 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.233231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.233992 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.234002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.234339 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.234341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.234435 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.234475 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:33.234680 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:33.234683 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:57:33.234723 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 02:57:33.234777 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":83,"threads_started":1}
I20260811 02:57:33.240180 13227 raft_consensus.cc:1261] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Refusing update from remote peer ffe2d38d01e64d7f98c969a1f3875f1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:33.240175 12975 raft_consensus.cc:1261] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Refusing update from remote peer ffe2d38d01e64d7f98c969a1f3875f1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:33.240583 13534 consensus_queue.cc:1035] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:33.240774 13544 consensus_queue.cc:1035] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:33.256810 13547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.253802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:48505 (local address 127.8.203.65:45845)
0811 02:57:33.253915 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:33.253919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:33.254165 (+ 246us) server_negotiation.cc:407] Connection header received
0811 02:57:33.254240 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:33.254242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:33.254280 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:33.254331 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:33.254490 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.254959 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.255778 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.256030 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.256117 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.256379 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:33.256456 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:33.256529 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:33.256646 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 02:57:33.256698 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:57:33.256834 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:33.253742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45845 (local address 127.8.203.67:48505)
0811 02:57:33.254088 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:33.254100 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:33.254180 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:33.254340 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:33.254342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:33.254347 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:57:33.254424 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.254428 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.255030 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.255032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.255670 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:33.255680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:33.256142 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:33.256145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:33.256230 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:33.256257 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:57:33.256542 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:57:33.256548 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:57:33.256615 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:57:33.256695 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":102,"threads_started":1}
I20260811 02:57:33.277516 12405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:33.277655 12405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:33.279484 12405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Sending DeleteTablet for 3 replicas of tablet 240546de0c3f4bddbdc881cc22a9cce5
I20260811 02:57:33.279985 12686 tablet_service.cc:1514] Processing DeleteTablet for tablet 240546de0c3f4bddbdc881cc22a9cce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:33 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:49200
I20260811 02:57:33.280031 12933 tablet_service.cc:1514] Processing DeleteTablet for tablet 240546de0c3f4bddbdc881cc22a9cce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:33 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:47910
I20260811 02:57:33.280157 13504 tablet_replica.cc:330] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: stopping tablet replica
I20260811 02:57:33.280171 13505 tablet_replica.cc:330] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: stopping tablet replica
I20260811 02:57:33.280264 13505 raft_consensus.cc:2229] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:33.280264 13504 raft_consensus.cc:2229] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:33.280368 13505 raft_consensus.cc:2258] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:33.280545 13504 raft_consensus.cc:2258] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:33.281001 13505 ts_tablet_manager.cc:1905] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:33.281352 13504 ts_tablet_manager.cc:1905] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:33.283396 13504 ts_tablet_manager.cc:1918] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:33.283468 13504 log.cc:1196] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/240546de0c3f4bddbdc881cc22a9cce5
I20260811 02:57:33.283679 13504 ts_tablet_manager.cc:1939] T 240546de0c3f4bddbdc881cc22a9cce5 P ffe2d38d01e64d7f98c969a1f3875f1c: Deleting consensus metadata
I20260811 02:57:33.283708 13505 ts_tablet_manager.cc:1918] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:33.283787 13505 log.cc:1196] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/240546de0c3f4bddbdc881cc22a9cce5
I20260811 02:57:33.284040 13505 ts_tablet_manager.cc:1939] T 240546de0c3f4bddbdc881cc22a9cce5 P b79939ab57af4ab68076e57f67865468: Deleting consensus metadata
I20260811 02:57:33.284237 12381 catalog_manager.cc:4841] TS ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845): tablet 240546de0c3f4bddbdc881cc22a9cce5 (table test-table [id=3e4fc1d47ae649b09d75742384d62553]) successfully deleted
I20260811 02:57:33.284545 12405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:33.284699 12385 catalog_manager.cc:4841] TS b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873): tablet 240546de0c3f4bddbdc881cc22a9cce5 (table test-table [id=3e4fc1d47ae649b09d75742384d62553]) successfully deleted
W20260811 02:57:33.284915 12405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:33.285558 13186 tablet_service.cc:1514] Processing DeleteTablet for tablet 240546de0c3f4bddbdc881cc22a9cce5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:33 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52796
I20260811 02:57:33.285729 13506 tablet_replica.cc:330] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: stopping tablet replica
I20260811 02:57:33.285859 13506 raft_consensus.cc:2229] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:33.285971 13506 raft_consensus.cc:2258] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:33.286679 13506 ts_tablet_manager.cc:1905] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:33.288514 13506 ts_tablet_manager.cc:1918] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:57:33.288576 13506 log.cc:1196] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/240546de0c3f4bddbdc881cc22a9cce5
I20260811 02:57:33.288769 13506 ts_tablet_manager.cc:1939] T 240546de0c3f4bddbdc881cc22a9cce5 P ca5c4446ed784f8caa74ce58d77e5119: Deleting consensus metadata
I20260811 02:57:33.289327 12383 catalog_manager.cc:4841] TS ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581): tablet 240546de0c3f4bddbdc881cc22a9cce5 (table test-table [id=3e4fc1d47ae649b09d75742384d62553]) successfully deleted
I20260811 02:57:33.291797 12933 tablet_service.cc:1467] Processing CreateTablet for tablet 104c35a442a04060bcb826d582119a09 (DEFAULT_TABLE table=test-table [id=b18b34bfb1884d408a3e6d38113ad5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:33.291797 13186 tablet_service.cc:1467] Processing CreateTablet for tablet 104c35a442a04060bcb826d582119a09 (DEFAULT_TABLE table=test-table [id=b18b34bfb1884d408a3e6d38113ad5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:33.292044 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 104c35a442a04060bcb826d582119a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:33.292044 12933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 104c35a442a04060bcb826d582119a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:33.294162 13524 tablet_bootstrap.cc:492] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: Bootstrap starting.
I20260811 02:57:33.294209 13523 tablet_bootstrap.cc:492] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: Bootstrap starting.
I20260811 02:57:33.294939 13523 tablet_bootstrap.cc:654] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:33.294998 13524 tablet_bootstrap.cc:654] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:33.295760 12686 tablet_service.cc:1467] Processing CreateTablet for tablet 104c35a442a04060bcb826d582119a09 (DEFAULT_TABLE table=test-table [id=b18b34bfb1884d408a3e6d38113ad5e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:33.296020 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 104c35a442a04060bcb826d582119a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:33.296080 13523 tablet_bootstrap.cc:492] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: No bootstrap required, opened a new log
I20260811 02:57:33.296159 13523 ts_tablet_manager.cc:1397] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:33.296520 13523 raft_consensus.cc:348] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.296627 13523 raft_consensus.cc:374] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:33.296662 13523 raft_consensus.cc:729] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79939ab57af4ab68076e57f67865468, State: Initialized, Role: FOLLOWER
I20260811 02:57:33.296782 13523 consensus_queue.cc:260] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.297051 13523 ts_tablet_manager.cc:1428] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:57:33.298097 13525 tablet_bootstrap.cc:492] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c: Bootstrap starting.
I20260811 02:57:33.298964 13525 tablet_bootstrap.cc:654] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:33.299719 13524 tablet_bootstrap.cc:492] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: No bootstrap required, opened a new log
I20260811 02:57:33.299799 13524 ts_tablet_manager.cc:1397] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 02:57:33.300060 13525 tablet_bootstrap.cc:492] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c: No bootstrap required, opened a new log
I20260811 02:57:33.300117 13525 ts_tablet_manager.cc:1397] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:33.300141 13524 raft_consensus.cc:348] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.300235 13524 raft_consensus.cc:374] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:33.300263 13524 raft_consensus.cc:729] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5c4446ed784f8caa74ce58d77e5119, State: Initialized, Role: FOLLOWER
I20260811 02:57:33.300382 13524 consensus_queue.cc:260] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.300470 13525 raft_consensus.cc:348] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.300568 13525 raft_consensus.cc:374] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:33.300601 13525 raft_consensus.cc:729] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe2d38d01e64d7f98c969a1f3875f1c, State: Initialized, Role: FOLLOWER
I20260811 02:57:33.300614 13524 ts_tablet_manager.cc:1428] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:57:33.300711 13525 consensus_queue.cc:260] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.300948 13525 ts_tablet_manager.cc:1428] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:33.536171 13572 raft_consensus.cc:482] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:33.536371 13572 raft_consensus.cc:504] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.536803 13572 leader_election.cc:288] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:33.537222 12974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "104c35a442a04060bcb826d582119a09" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79939ab57af4ab68076e57f67865468" is_pre_election: true
I20260811 02:57:33.537233 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "104c35a442a04060bcb826d582119a09" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119" is_pre_election: true
I20260811 02:57:33.537410 12974 raft_consensus.cc:2454] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 0.
I20260811 02:57:33.537417 13227 raft_consensus.cc:2454] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 0.
I20260811 02:57:33.537738 12586 leader_election.cc:302] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b79939ab57af4ab68076e57f67865468, ca5c4446ed784f8caa74ce58d77e5119, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:33.537921 13572 raft_consensus.cc:2781] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:33.537976 13572 raft_consensus.cc:482] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:33.538000 13572 raft_consensus.cc:3037] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:33.539013 13572 raft_consensus.cc:504] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.539328 13572 leader_election.cc:288] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 election: Requested vote from peers b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873), ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581)
I20260811 02:57:33.539675 13227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "104c35a442a04060bcb826d582119a09" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5c4446ed784f8caa74ce58d77e5119"
I20260811 02:57:33.539783 13227 raft_consensus.cc:3037] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:33.539855 12974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "104c35a442a04060bcb826d582119a09" candidate_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79939ab57af4ab68076e57f67865468"
I20260811 02:57:33.540005 12974 raft_consensus.cc:3037] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:33.540858 13227 raft_consensus.cc:2454] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 1.
I20260811 02:57:33.541093 12974 raft_consensus.cc:2454] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe2d38d01e64d7f98c969a1f3875f1c in term 1.
I20260811 02:57:33.541115 12579 leader_election.cc:302] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca5c4446ed784f8caa74ce58d77e5119, ffe2d38d01e64d7f98c969a1f3875f1c; no voters:
I20260811 02:57:33.541319 13572 raft_consensus.cc:2781] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:33.541419 13572 raft_consensus.cc:686] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [term 1 LEADER]: Becoming Leader. State: Replica: ffe2d38d01e64d7f98c969a1f3875f1c, State: Running, Role: LEADER
I20260811 02:57:33.541559 13572 consensus_queue.cc:237] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } }
I20260811 02:57:33.543213 12405 catalog_manager.cc:5495] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65). New cstate: current_term: 1 leader_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe2d38d01e64d7f98c969a1f3875f1c" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260811 02:57:33.598964 13227 raft_consensus.cc:1261] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Refusing update from remote peer ffe2d38d01e64d7f98c969a1f3875f1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:33.598964 12974 raft_consensus.cc:1261] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Refusing update from remote peer ffe2d38d01e64d7f98c969a1f3875f1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:33.599342 13572 consensus_queue.cc:1035] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79939ab57af4ab68076e57f67865468" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:33.599507 13534 consensus_queue.cc:1035] T 104c35a442a04060bcb826d582119a09 P ffe2d38d01e64d7f98c969a1f3875f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5c4446ed784f8caa74ce58d77e5119" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:57:33.610908 13465 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 02:57:35.159307 12405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:35.159461 12405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:35.161727 12405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20ec90b4e9d3466a982fc0efc96426d4: Sending DeleteTablet for 3 replicas of tablet 104c35a442a04060bcb826d582119a09
I20260811 02:57:35.162214 12686 tablet_service.cc:1514] Processing DeleteTablet for tablet 104c35a442a04060bcb826d582119a09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:35 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:49200
I20260811 02:57:35.162319 13183 tablet_service.cc:1514] Processing DeleteTablet for tablet 104c35a442a04060bcb826d582119a09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:35 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52796
I20260811 02:57:35.162379 12933 tablet_service.cc:1514] Processing DeleteTablet for tablet 104c35a442a04060bcb826d582119a09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:35 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:47910
I20260811 02:57:35.162706 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 12547
I20260811 02:57:35.170928 13631 tablet_replica.cc:330] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: stopping tablet replica
I20260811 02:57:35.171115 13631 raft_consensus.cc:2229] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:35.171255 13631 raft_consensus.cc:2258] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:35.162765 13633 tablet_replica.cc:330] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: stopping tablet replica
I20260811 02:57:35.172621 13633 raft_consensus.cc:2229] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:35.172782 13633 raft_consensus.cc:2258] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:35.172916 13631 ts_tablet_manager.cc:1905] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:35.175405 13631 ts_tablet_manager.cc:1918] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:57:35.175478 13631 log.cc:1196] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/104c35a442a04060bcb826d582119a09
I20260811 02:57:35.175678 13631 ts_tablet_manager.cc:1939] T 104c35a442a04060bcb826d582119a09 P ca5c4446ed784f8caa74ce58d77e5119: Deleting consensus metadata
I20260811 02:57:35.176433 12383 catalog_manager.cc:4841] TS ca5c4446ed784f8caa74ce58d77e5119 (127.8.203.67:41581): tablet 104c35a442a04060bcb826d582119a09 (table test-table [id=b18b34bfb1884d408a3e6d38113ad5e8]) successfully deleted
I20260811 02:57:35.179284 13633 ts_tablet_manager.cc:1905] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:35.181383 13633 ts_tablet_manager.cc:1918] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:57:35.181457 13633 log.cc:1196] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/104c35a442a04060bcb826d582119a09
I20260811 02:57:35.181667 13633 ts_tablet_manager.cc:1939] T 104c35a442a04060bcb826d582119a09 P b79939ab57af4ab68076e57f67865468: Deleting consensus metadata
I20260811 02:57:35.182403 12385 catalog_manager.cc:4841] TS b79939ab57af4ab68076e57f67865468 (127.8.203.66:41873): tablet 104c35a442a04060bcb826d582119a09 (table test-table [id=b18b34bfb1884d408a3e6d38113ad5e8]) successfully deleted
W20260811 02:57:35.186502 12381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:45845 (error 108)
I20260811 02:57:35.186851 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 12828
I20260811 02:57:35.193568 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:35.193053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45845 (local address 127.0.0.1:49230)
0811 02:57:35.193368 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:35.193457 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":87,"threads_started":1}
W20260811 02:57:35.193805 12381 catalog_manager.cc:4584] TS ffe2d38d01e64d7f98c969a1f3875f1c (127.8.203.65:45845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 104c35a442a04060bcb826d582119a09: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45845: connect: Connection refused (error 111)
I20260811 02:57:35.199366 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 13063
I20260811 02:57:35.209738 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 12356
2026-08-11T02:57:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6579 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:35 dist-test-slave-gjw7 krb5kdc[13641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:35 dist-test-slave-gjw7 krb5kdc[13641](info): set up 2 sockets
Aug 11 02:57:35 dist-test-slave-gjw7 krb5kdc[13641](info): commencing operation
krb5kdc: starting...
W20260811 02:57:36.418118 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:36 dist-test-slave-gjw7 krb5kdc[13641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:36.532699 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:44653
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:44653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:36.644197 13660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:36.644474 13660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:36.644541 13660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:36.648517 13660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:36.648581 13660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:36.648613 13660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:36.648641 13660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:36.648676 13660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:36.653836 13660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:44653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:44653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:36.654841 13660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:36.655871 13660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:36.664438 13660 server_base.cc:1034] running on GCE node
W20260811 02:57:36.664372 13679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:36.664395 13678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:36.664398 13684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:36.665194 13660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:36.666270 13660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:36.667421 13660 hybrid_clock.cc:648] HybridClock initialized: now 1786417056667398 us; error 38 us; skew 500 ppm
Aug 11 02:57:36 dist-test-slave-gjw7 krb5kdc[13641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417056, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:36.670002 13660 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:36.670836 13660 webserver.cc:460] Webserver started at http://127.8.203.126:39705/ using document root <none> and password file <none>
I20260811 02:57:36.671366 13660 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:36.671432 13660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:36.671636 13660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:36.673650 13660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "5af258fa196c4e1ab015684f8f418870"
format_stamp: "Formatted at 2026-08-11 02:57:36 on dist-test-slave-gjw7"
server_key: "2d8eb0487434b36a5b4cb5defd860251"
server_key_iv: "86259e413e86a1bd2dfcf5761e7f0572"
server_key_version: "encryptionkey@0"
I20260811 02:57:36.674201 13660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "5af258fa196c4e1ab015684f8f418870"
format_stamp: "Formatted at 2026-08-11 02:57:36 on dist-test-slave-gjw7"
server_key: "2d8eb0487434b36a5b4cb5defd860251"
server_key_iv: "86259e413e86a1bd2dfcf5761e7f0572"
server_key_version: "encryptionkey@0"
I20260811 02:57:36.677752 13660 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:57:36.679943 13700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:36.681069 13660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:36.681180 13660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "5af258fa196c4e1ab015684f8f418870"
format_stamp: "Formatted at 2026-08-11 02:57:36 on dist-test-slave-gjw7"
server_key: "2d8eb0487434b36a5b4cb5defd860251"
server_key_iv: "86259e413e86a1bd2dfcf5761e7f0572"
server_key_version: "encryptionkey@0"
I20260811 02:57:36.681268 13660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:36.697422 13660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:36.699939 13660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:36.700096 13660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:36.706611 13660 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:44653
I20260811 02:57:36.706619 13793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:44653 every 8 connection(s)
I20260811 02:57:36.707491 13660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:36.707854 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 13660
I20260811 02:57:36.707965 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:36.708199 9005 external_mini_cluster.cc:1428] Setting key 07a49a625e1e994071669ff4d7ac287b
I20260811 02:57:36.710620 13794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:36.716908 13794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870: Bootstrap starting.
Aug 11 02:57:36 dist-test-slave-gjw7 krb5kdc[13641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:36.719502 13794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:36.720360 13794 log.cc:824] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:36.722327 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:36.709577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51398 (local address 127.8.203.126:44653)
0811 02:57:36.709977 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:36.709997 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:36.710039 (+ 42us) server_negotiation.cc:407] Connection header received
0811 02:57:36.710854 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:36.710877 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:36.711293 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:36.711650 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:36.712279 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:36.713748 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:36.714544 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:36.715201 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:36.715365 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:36.717565 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:36.717571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:36.717586 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:36.717616 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:36.719373 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:36.719787 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:36.719793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:36.719799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:36.719936 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:36.720165 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:36.720168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:36.720169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:36.720671 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:36.720839 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:36.721114 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 02:57:36.721444 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260811 02:57:36.722561 13794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870: No bootstrap required, opened a new log
I20260811 02:57:36.726163 13794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af258fa196c4e1ab015684f8f418870" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44653 } }
I20260811 02:57:36.726372 13794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:36.726423 13794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af258fa196c4e1ab015684f8f418870, State: Initialized, Role: FOLLOWER
I20260811 02:57:36.726775 13794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af258fa196c4e1ab015684f8f418870" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44653 } }
I20260811 02:57:36.726887 13794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:36.726930 13794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:36.726997 13794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:36.727766 13794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af258fa196c4e1ab015684f8f418870" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44653 } }
I20260811 02:57:36.728051 13794 leader_election.cc:302] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af258fa196c4e1ab015684f8f418870; no voters:
I20260811 02:57:36.728281 13794 leader_election.cc:288] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:36.728399 13800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:36.728636 13800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [term 1 LEADER]: Becoming Leader. State: Replica: 5af258fa196c4e1ab015684f8f418870, State: Running, Role: LEADER
I20260811 02:57:36.728922 13800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af258fa196c4e1ab015684f8f418870" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44653 } }
I20260811 02:57:36.729411 13794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:36.730476 13801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af258fa196c4e1ab015684f8f418870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af258fa196c4e1ab015684f8f418870" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44653 } } }
I20260811 02:57:36.730607 13801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:36.730911 13802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af258fa196c4e1ab015684f8f418870. Latest consensus state: current_term: 1 leader_uuid: "5af258fa196c4e1ab015684f8f418870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af258fa196c4e1ab015684f8f418870" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44653 } } }
I20260811 02:57:36.730999 13802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:36.731024 13812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:36.734081 13812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:36.739517 13812 catalog_manager.cc:1347] Generated new cluster ID: 9fa2e41c55c84467842ff5ced924d046
I20260811 02:57:36.739576 13812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:36.769806 13812 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:36.770998 13812 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:36.780151 13812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af258fa196c4e1ab015684f8f418870: Generated new TSK 0
I20260811 02:57:36.780901 13812 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:36.813124 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:44653
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:36.921437 13831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:36.921650 13831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:36.921689 13831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:36.925525 13831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:36.925570 13831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:36.925622 13831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:36.930631 13831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:44653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:36.931499 13831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:36.932426 13831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:36.941367 13845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:36.941365 13848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:36.941439 13831 server_base.cc:1034] running on GCE node
W20260811 02:57:36.941371 13842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:36.941922 13831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:36.942504 13831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:36.943655 13831 hybrid_clock.cc:648] HybridClock initialized: now 1786417056943639 us; error 27 us; skew 500 ppm
Aug 11 02:57:36 dist-test-slave-gjw7 krb5kdc[13641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417056, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:36.946244 13831 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:36.947028 13831 webserver.cc:460] Webserver started at http://127.8.203.65:46631/ using document root <none> and password file <none>
I20260811 02:57:36.947511 13831 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:36.947566 13831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:36.947726 13831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:36.949657 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "5e6510c3afbf488ab7e98fa7489dd5a1"
format_stamp: "Formatted at 2026-08-11 02:57:36 on dist-test-slave-gjw7"
server_key: "6330c7e91c384756649ca730bee0b28f"
server_key_iv: "abed3436d6f8368ac94714e3bd79e59a"
server_key_version: "encryptionkey@0"
I20260811 02:57:36.950155 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "5e6510c3afbf488ab7e98fa7489dd5a1"
format_stamp: "Formatted at 2026-08-11 02:57:36 on dist-test-slave-gjw7"
server_key: "6330c7e91c384756649ca730bee0b28f"
server_key_iv: "abed3436d6f8368ac94714e3bd79e59a"
server_key_version: "encryptionkey@0"
I20260811 02:57:36.953563 13831 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:57:36.955829 13860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:36.957015 13831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:36.957108 13831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "5e6510c3afbf488ab7e98fa7489dd5a1"
format_stamp: "Formatted at 2026-08-11 02:57:36 on dist-test-slave-gjw7"
server_key: "6330c7e91c384756649ca730bee0b28f"
server_key_iv: "abed3436d6f8368ac94714e3bd79e59a"
server_key_version: "encryptionkey@0"
I20260811 02:57:36.957201 13831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:36.977172 13831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:36.980029 13831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:36.980185 13831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:36.980700 13831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:36.981494 13831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:36.981539 13831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:36.981592 13831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:36.981621 13831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:36.989949 13831 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:35337
I20260811 02:57:36.989980 14099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:35337 every 8 connection(s)
I20260811 02:57:36.990731 13831 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:36.998799 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 13831
I20260811 02:57:36.998893 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
Aug 11 02:57:36 dist-test-slave-gjw7 krb5kdc[13641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417056, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:36.999158 9005 external_mini_cluster.cc:1428] Setting key 491aedc336126d7c4eb68d1a94ca98a5
I20260811 02:57:37.002926 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:36.992338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:47337 (local address 127.8.203.126:44653)
0811 02:57:36.992459 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:36.992462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:36.993228 (+ 766us) server_negotiation.cc:407] Connection header received
0811 02:57:36.994428 (+ 1200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:36.994431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:36.994477 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:36.994545 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:36.995576 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:36.996103 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:36.997115 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:36.997367 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:36.997449 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:36.999577 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:36.999581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:36.999583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:36.999607 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:37.000873 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:37.001438 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:37.001441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:37.001443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:37.001517 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:37.001831 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:37.001833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:37.001834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:37.001994 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:37.002072 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:37.002636 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 02:57:37.002761 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:57:37.003575 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:36.992666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:44653 (local address 127.8.203.65:47337)
0811 02:57:36.993081 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:36.993121 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:36.994244 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:36.994689 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:36.994695 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:36.995024 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:36.995464 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:36.995477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:36.996186 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:36.996188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:36.997029 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:36.997038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:36.997477 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:36.997479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:36.997608 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:36.998016 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:36.998032 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:36.999424 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:37.000985 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:37.000990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:37.001003 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:37.001350 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:37.001588 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:37.001590 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:37.001592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:37.001757 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:37.002076 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:37.002082 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:37.002369 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 02:57:37.002559 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 02:57:37.002787 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
I20260811 02:57:37.004618 14100 heartbeater.cc:347] Connected to a master server at 127.8.203.126:44653
I20260811 02:57:37.004830 14100 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:37.005254 14100 heartbeater.cc:511] Master 127.8.203.126:44653 requested a full tablet report, sending...
I20260811 02:57:37.006834 13726 ts_manager.cc:194] Registered new tserver with Master: 5e6510c3afbf488ab7e98fa7489dd5a1 (127.8.203.65:35337)
I20260811 02:57:37.008364 13726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:47337
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:37.024945 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:44653
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:37.134052 14108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:37.134289 14108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:37.134327 14108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:37.138304 14108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:37.138355 14108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:37.138408 14108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:37.143388 14108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:44653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:37.144294 14108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:37.145210 14108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:37.154491 14108 server_base.cc:1034] running on GCE node
W20260811 02:57:37.154457 14117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:37.154457 14115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:37.154453 14119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:37.155038 14108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:37.155618 14108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:37.156760 14108 hybrid_clock.cc:648] HybridClock initialized: now 1786417057156735 us; error 38 us; skew 500 ppm
Aug 11 02:57:37 dist-test-slave-gjw7 krb5kdc[13641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:37.159348 14108 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:37.160197 14108 webserver.cc:460] Webserver started at http://127.8.203.66:44527/ using document root <none> and password file <none>
I20260811 02:57:37.160720 14108 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:37.160784 14108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:37.160984 14108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:37.162983 14108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "f1bcaeee2e094fe69cc2764a110d44d0"
format_stamp: "Formatted at 2026-08-11 02:57:37 on dist-test-slave-gjw7"
server_key: "9bb71b3108e45d5f07f92a17acbcd661"
server_key_iv: "078f81eb2a25cb0a9d0ffaf42cb74648"
server_key_version: "encryptionkey@0"
I20260811 02:57:37.163506 14108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "f1bcaeee2e094fe69cc2764a110d44d0"
format_stamp: "Formatted at 2026-08-11 02:57:37 on dist-test-slave-gjw7"
server_key: "9bb71b3108e45d5f07f92a17acbcd661"
server_key_iv: "078f81eb2a25cb0a9d0ffaf42cb74648"
server_key_version: "encryptionkey@0"
I20260811 02:57:37.167040 14108 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:37.169305 14134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:37.170584 14108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:37.170696 14108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "f1bcaeee2e094fe69cc2764a110d44d0"
format_stamp: "Formatted at 2026-08-11 02:57:37 on dist-test-slave-gjw7"
server_key: "9bb71b3108e45d5f07f92a17acbcd661"
server_key_iv: "078f81eb2a25cb0a9d0ffaf42cb74648"
server_key_version: "encryptionkey@0"
I20260811 02:57:37.170799 14108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:37.179174 14108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:37.182083 14108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:37.182243 14108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:37.182775 14108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:37.183526 14108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:37.183570 14108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:37.183619 14108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:37.183648 14108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:37.191636 14108 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:40309
I20260811 02:57:37.191658 14328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:40309 every 8 connection(s)
I20260811 02:57:37.192436 14108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:57:37 dist-test-slave-gjw7 krb5kdc[13641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:37.201426 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 14108
I20260811 02:57:37.201538 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:37.201860 9005 external_mini_cluster.cc:1428] Setting key b19d311b22ce77752dd3003d8696fc4b
I20260811 02:57:37.204890 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:37.194102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:53459 (local address 127.8.203.126:44653)
0811 02:57:37.194247 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:37.194252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:37.195016 (+ 764us) server_negotiation.cc:407] Connection header received
0811 02:57:37.196207 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:37.196210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:37.196258 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:37.196325 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:37.197402 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.197917 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.198987 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.199208 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.199294 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:37.201445 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:37.201450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:37.201453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:37.201482 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:37.202965 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:37.203479 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:37.203482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:37.203484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:37.203569 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:37.203864 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:37.203867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:37.203869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:37.204057 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:37.204141 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:37.204606 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 02:57:37.204726 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:57:37.205828 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:37.194465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:44653 (local address 127.8.203.66:53459)
0811 02:57:37.194876 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:37.194917 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:37.196024 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:37.196473 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:37.196480 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:37.196796 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:37.197244 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:37.197258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.198011 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.198013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:37.198858 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:37.198867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.199324 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.199326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:37.199464 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:37.199908 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:37.199922 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:37.201260 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:37.203076 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:37.203081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:37.203096 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:37.203386 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:37.203647 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:37.203650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:37.203653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:37.203779 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:37.204153 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:37.204159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:37.204387 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 02:57:37.204545 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 02:57:37.204837 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":103,"threads_started":1}
I20260811 02:57:37.206822 14330 heartbeater.cc:347] Connected to a master server at 127.8.203.126:44653
I20260811 02:57:37.207047 14330 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:37.207563 14330 heartbeater.cc:511] Master 127.8.203.126:44653 requested a full tablet report, sending...
I20260811 02:57:37.208611 13726 ts_manager.cc:194] Registered new tserver with Master: f1bcaeee2e094fe69cc2764a110d44d0 (127.8.203.66:40309)
I20260811 02:57:37.209133 13726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:53459
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:37.228029 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:44653
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:37.335692 14341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:37.335938 14341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:37.335986 14341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:37.339882 14341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:37.339926 14341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:37.339991 14341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:37.344973 14341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:44653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:37.345949 14341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:37.346890 14341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:37.355966 14348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:37.355991 14341 server_base.cc:1034] running on GCE node
W20260811 02:57:37.355931 14351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:37.355937 14355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:37.356542 14341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:37.357079 14341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:37.358227 14341 hybrid_clock.cc:648] HybridClock initialized: now 1786417057358201 us; error 37 us; skew 500 ppm
Aug 11 02:57:37 dist-test-slave-gjw7 krb5kdc[13641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:37.360775 14341 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:37.361617 14341 webserver.cc:460] Webserver started at http://127.8.203.67:35575/ using document root <none> and password file <none>
I20260811 02:57:37.362102 14341 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:37.362155 14341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:37.362315 14341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:37.364506 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "d86c1d6b36dc4c119d68ae14cf6f1d23"
format_stamp: "Formatted at 2026-08-11 02:57:37 on dist-test-slave-gjw7"
server_key: "c5eb2e5842c684f8eb876c0e8b172318"
server_key_iv: "aee4a1ffd6fc437e85916644d7faf74e"
server_key_version: "encryptionkey@0"
I20260811 02:57:37.365029 14341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "d86c1d6b36dc4c119d68ae14cf6f1d23"
format_stamp: "Formatted at 2026-08-11 02:57:37 on dist-test-slave-gjw7"
server_key: "c5eb2e5842c684f8eb876c0e8b172318"
server_key_iv: "aee4a1ffd6fc437e85916644d7faf74e"
server_key_version: "encryptionkey@0"
I20260811 02:57:37.368533 14341 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:37.370960 14366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:37.372093 14341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:37.372193 14341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "d86c1d6b36dc4c119d68ae14cf6f1d23"
format_stamp: "Formatted at 2026-08-11 02:57:37 on dist-test-slave-gjw7"
server_key: "c5eb2e5842c684f8eb876c0e8b172318"
server_key_iv: "aee4a1ffd6fc437e85916644d7faf74e"
server_key_version: "encryptionkey@0"
I20260811 02:57:37.372294 14341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:37.392160 14341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:37.394829 14341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:37.395000 14341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:37.395538 14341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:37.396332 14341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:37.396379 14341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:37.396422 14341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:37.396450 14341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:37.404585 14341 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:46025
I20260811 02:57:37.404616 14598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:46025 every 8 connection(s)
I20260811 02:57:37.405426 14341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:37.413542 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 14341
I20260811 02:57:37.413661 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:37.413964 9005 external_mini_cluster.cc:1428] Setting key efc1047268ecaed2c1ad4624a13d0932
Aug 11 02:57:37 dist-test-slave-gjw7 krb5kdc[13641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:37.418160 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:37.407086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:34279 (local address 127.8.203.126:44653)
0811 02:57:37.407221 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:37.407225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:37.408011 (+ 786us) server_negotiation.cc:407] Connection header received
0811 02:57:37.409253 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:37.409256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:37.409353 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:37.409429 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:37.410543 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.411065 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.412170 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.412405 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.412492 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:37.414947 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:37.414950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:37.414952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:37.414974 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:37.416355 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:37.416791 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:37.416794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:37.416795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:37.416878 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:37.417125 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:37.417127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:37.417128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:37.417280 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:37.417384 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:37.417887 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 02:57:37.418003 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:57:37.418874 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:37.407433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:44653 (local address 127.8.203.67:34279)
0811 02:57:37.407870 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:37.407911 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:37.409043 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:37.409609 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:37.409616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:37.409953 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:37.410405 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:37.410419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.411187 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.411190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:37.412050 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:37.412060 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:37.412543 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:37.412547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:37.412701 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:37.413157 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:37.413172 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:37.414741 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:37.416463 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:37.416468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:37.416479 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:37.416719 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:37.416942 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:37.416944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:37.416945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:37.417062 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:37.417420 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:37.417426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:37.417646 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 02:57:37.417812 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 02:57:37.418111 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260811 02:57:37.419849 14599 heartbeater.cc:347] Connected to a master server at 127.8.203.126:44653
I20260811 02:57:37.420058 14599 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:37.420508 14599 heartbeater.cc:511] Master 127.8.203.126:44653 requested a full tablet report, sending...
I20260811 02:57:37.421468 13726 ts_manager.cc:194] Registered new tserver with Master: d86c1d6b36dc4c119d68ae14cf6f1d23 (127.8.203.67:46025)
I20260811 02:57:37.422005 13726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:34279
I20260811 02:57:37.427551 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:57:37.442063 13798 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:37.440731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.8.203.126:44653)
0811 02:57:37.440874 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:37.440879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:37.441024 (+ 145us) server_negotiation.cc:407] Connection header received
0811 02:57:37.441132 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:37.441135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:37.441176 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:37.441240 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:37.441926 (+ 686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51404: BlockingRecv error: recv got EOF from 127.0.0.1:51404 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:57:37.443305 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 13831
I20260811 02:57:37.479100 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 14108
I20260811 02:57:37.492061 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 14341
I20260811 02:57:37.508471 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 13660
2026-08-11T02:57:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2297 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:37 dist-test-slave-gjw7 krb5kdc[14631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:37 dist-test-slave-gjw7 krb5kdc[14631](info): set up 2 sockets
Aug 11 02:57:37 dist-test-slave-gjw7 krb5kdc[14631](info): commencing operation
krb5kdc: starting...
W20260811 02:57:38.734432 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:38 dist-test-slave-gjw7 krb5kdc[14631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:38.849536 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:39201
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:43765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:39201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:38.957973 14649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:38.958211 14649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:38.958252 14649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:38.962453 14649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:38.962503 14649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:38.962522 14649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:38.962538 14649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:38.962565 14649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:38.967594 14649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:43765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:39201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:39201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:38.968684 14649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:38.969761 14649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:38.977856 14661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:38.977854 14668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:38.977887 14657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:38.978000 14649 server_base.cc:1034] running on GCE node
I20260811 02:57:38.978749 14649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:38.979785 14649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:38.980944 14649 hybrid_clock.cc:648] HybridClock initialized: now 1786417058980926 us; error 41 us; skew 500 ppm
Aug 11 02:57:38 dist-test-slave-gjw7 krb5kdc[14631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417058, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:38.987408 14649 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:38.988242 14649 webserver.cc:460] Webserver started at http://127.8.203.126:39641/ using document root <none> and password file <none>
I20260811 02:57:38.988760 14649 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:38.988834 14649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:38.989039 14649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:38.991092 14649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "1b3ecd44aac64604b602874680ceb267"
format_stamp: "Formatted at 2026-08-11 02:57:38 on dist-test-slave-gjw7"
server_key: "11c37ca54f0f4cf3ab332a762d53fabf"
server_key_iv: "c4330963ebc78444a50748fb4ec7b977"
server_key_version: "encryptionkey@0"
I20260811 02:57:38.991629 14649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "1b3ecd44aac64604b602874680ceb267"
format_stamp: "Formatted at 2026-08-11 02:57:38 on dist-test-slave-gjw7"
server_key: "11c37ca54f0f4cf3ab332a762d53fabf"
server_key_iv: "c4330963ebc78444a50748fb4ec7b977"
server_key_version: "encryptionkey@0"
I20260811 02:57:38.995154 14649 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:38.997401 14683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:38.998503 14649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:38.998612 14649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "1b3ecd44aac64604b602874680ceb267"
format_stamp: "Formatted at 2026-08-11 02:57:38 on dist-test-slave-gjw7"
server_key: "11c37ca54f0f4cf3ab332a762d53fabf"
server_key_iv: "c4330963ebc78444a50748fb4ec7b977"
server_key_version: "encryptionkey@0"
I20260811 02:57:38.998700 14649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:39.011909 14649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:39.014426 14649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:39.014587 14649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:39.020998 14649 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:39201
I20260811 02:57:39.021247 14794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:39201 every 8 connection(s)
I20260811 02:57:39.021955 14649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:39.024442 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 14649
I20260811 02:57:39.024562 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:39.024686 14797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:39.024865 9005 external_mini_cluster.cc:1428] Setting key 3be9568f652566d98119005c0779d095
I20260811 02:57:39.031410 14797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267: Bootstrap starting.
Aug 11 02:57:39 dist-test-slave-gjw7 krb5kdc[14631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:39.034293 14797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:39.035233 14797 log.cc:824] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:39.037439 14797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267: No bootstrap required, opened a new log
I20260811 02:57:39.039563 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:39.026129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.8.203.126:39201)
0811 02:57:39.026518 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:39.026534 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:39.026569 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:57:39.027309 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:39.027338 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:39.027654 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:39.027917 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:39.028789 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:39.029801 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:39.030926 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:39.031373 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:39.031481 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:39.034491 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:39.034499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:39.034516 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:39.034550 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:39.036357 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:39.036790 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:39.036795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:39.036801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:39.036940 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:39.037222 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:39.037226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:39.037227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:39.037750 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:39.037948 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:39.038311 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 02:57:39.038598 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260811 02:57:39.041369 14797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3ecd44aac64604b602874680ceb267" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39201 } }
I20260811 02:57:39.041604 14797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:39.041661 14797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3ecd44aac64604b602874680ceb267, State: Initialized, Role: FOLLOWER
I20260811 02:57:39.042120 14797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3ecd44aac64604b602874680ceb267" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39201 } }
I20260811 02:57:39.042285 14797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:39.042351 14797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:39.042383 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:57:39.042450 14797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:39.043305 14797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3ecd44aac64604b602874680ceb267" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39201 } }
I20260811 02:57:39.043627 14797 leader_election.cc:302] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b3ecd44aac64604b602874680ceb267; no voters:
I20260811 02:57:39.043926 14797 leader_election.cc:288] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:39.044026 14815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:39.044198 14815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [term 1 LEADER]: Becoming Leader. State: Replica: 1b3ecd44aac64604b602874680ceb267, State: Running, Role: LEADER
I20260811 02:57:39.044502 14815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3ecd44aac64604b602874680ceb267" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39201 } }
I20260811 02:57:39.045190 14797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:39.045703 14815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b3ecd44aac64604b602874680ceb267. Latest consensus state: current_term: 1 leader_uuid: "1b3ecd44aac64604b602874680ceb267" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3ecd44aac64604b602874680ceb267" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39201 } } }
I20260811 02:57:39.045816 14815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:39.046123 14816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b3ecd44aac64604b602874680ceb267" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3ecd44aac64604b602874680ceb267" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39201 } } }
I20260811 02:57:39.046185 14825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:39.046208 14816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:39.049516 14825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:39.055009 14825 catalog_manager.cc:1347] Generated new cluster ID: 4c8db5e297024a32a3245c9c4dbe605e
I20260811 02:57:39.055073 14825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:39.064157 14825 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:39.064970 14825 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:39.067755 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:39.061270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.8.203.126:39201)
0811 02:57:39.061399 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:39.061404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:39.061678 (+ 274us) server_negotiation.cc:407] Connection header received
0811 02:57:39.061802 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:39.061805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:39.061857 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:39.061950 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:39.062539 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:39.063137 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:39.065416 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:39.065771 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:39.065876 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:39.066764 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:39.066769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:39.066772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:39.067316 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:39.067351 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:39.067371 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:39.067457 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:39.067541 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 02:57:39.067618 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:57:39.074182 14825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b3ecd44aac64604b602874680ceb267: Generated new TSK 0
I20260811 02:57:39.074954 14825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:39.080015 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 14649
2026-08-11T02:57:39Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1586 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:39 dist-test-slave-gjw7 krb5kdc[14876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:39 dist-test-slave-gjw7 krb5kdc[14876](info): set up 2 sockets
Aug 11 02:57:39 dist-test-slave-gjw7 krb5kdc[14876](info): commencing operation
krb5kdc: starting...
W20260811 02:57:40.311345 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:40 dist-test-slave-gjw7 krb5kdc[14876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:40.425971 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46437
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:40.533263 14906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:40.533527 14906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:40.533578 14906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:40.537449 14906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:40.537501 14906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:40.537528 14906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:40.537549 14906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:40.537568 14906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:40.542591 14906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:40.543550 14906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:40.544504 14906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:40.552368 14913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:40.552490 14906 server_base.cc:1034] running on GCE node
W20260811 02:57:40.552495 14914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:40.552392 14916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:40.553201 14906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:40.554247 14906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:40.555403 14906 hybrid_clock.cc:648] HybridClock initialized: now 1786417060555388 us; error 27 us; skew 500 ppm
Aug 11 02:57:40 dist-test-slave-gjw7 krb5kdc[14876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:40.558130 14906 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:40.558986 14906 webserver.cc:460] Webserver started at http://127.8.203.126:42983/ using document root <none> and password file <none>
I20260811 02:57:40.559506 14906 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:40.559564 14906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:40.559760 14906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:40.561713 14906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "ce60fbb8471d44eaa0e1369a8c6386ca"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-gjw7"
server_key: "5b865f06599b4381075dd6d74596f81b"
server_key_iv: "be95350ddced4e6486d054cea183697d"
server_key_version: "encryptionkey@0"
I20260811 02:57:40.562242 14906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "ce60fbb8471d44eaa0e1369a8c6386ca"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-gjw7"
server_key: "5b865f06599b4381075dd6d74596f81b"
server_key_iv: "be95350ddced4e6486d054cea183697d"
server_key_version: "encryptionkey@0"
I20260811 02:57:40.565722 14906 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:40.567920 14934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:40.569051 14906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:40.569161 14906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "ce60fbb8471d44eaa0e1369a8c6386ca"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-gjw7"
server_key: "5b865f06599b4381075dd6d74596f81b"
server_key_iv: "be95350ddced4e6486d054cea183697d"
server_key_version: "encryptionkey@0"
I20260811 02:57:40.569249 14906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:40.585636 14906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:40.588584 14906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:40.588754 14906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:40.595428 14906 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:46437
I20260811 02:57:40.595685 15044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:46437 every 8 connection(s)
I20260811 02:57:40.596350 14906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:40.599074 15046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:40.601140 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 14906
I20260811 02:57:40.601254 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:40.601524 9005 external_mini_cluster.cc:1428] Setting key 71ac752c73b169ab2d77fcfd6fbcd231
I20260811 02:57:40.604770 15046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca: Bootstrap starting.
I20260811 02:57:40.607251 15046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:40.608026 15046 log.cc:824] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:57:40 dist-test-slave-gjw7 krb5kdc[14876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:40.610172 15046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca: No bootstrap required, opened a new log
I20260811 02:57:40.614545 15046 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46437 } }
I20260811 02:57:40.614818 15046 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:40.614873 15046 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce60fbb8471d44eaa0e1369a8c6386ca, State: Initialized, Role: FOLLOWER
I20260811 02:57:40.615366 15046 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46437 } }
I20260811 02:57:40.615527 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:40.602833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.8.203.126:46437)
0811 02:57:40.603229 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:40.603245 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:40.603280 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:57:40.603903 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:40.603920 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:40.604234 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:40.604500 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:40.605413 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:40.606374 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:40.607504 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:40.607965 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:40.608077 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:40.610658 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:40.610664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:40.610680 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:40.610710 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:40.612516 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:40.612923 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:40.612929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:40.612935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:40.613071 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:40.613341 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:40.613344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:40.613346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:40.613884 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:40.614089 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:40.614286 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 02:57:40.614572 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":118,"threads_started":1}
I20260811 02:57:40.615535 15046 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:40.615619 15046 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:40.615702 15046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:40.616772 15046 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46437 } }
I20260811 02:57:40.617100 15046 leader_election.cc:302] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce60fbb8471d44eaa0e1369a8c6386ca; no voters:
I20260811 02:57:40.617441 15046 leader_election.cc:288] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:40.617515 15053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:40.617720 15053 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [term 1 LEADER]: Becoming Leader. State: Replica: ce60fbb8471d44eaa0e1369a8c6386ca, State: Running, Role: LEADER
I20260811 02:57:40.618088 15053 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46437 } }
I20260811 02:57:40.618582 15046 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:40.619164 15057 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce60fbb8471d44eaa0e1369a8c6386ca. Latest consensus state: current_term: 1 leader_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46437 } } }
I20260811 02:57:40.619292 15057 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:40.619620 15056 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce60fbb8471d44eaa0e1369a8c6386ca" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46437 } } }
I20260811 02:57:40.619732 15056 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:40.620002 15067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:40.623028 15067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:40.628310 15067 catalog_manager.cc:1347] Generated new cluster ID: 0400f8b2efb2470caf146e8e8169ad0c
I20260811 02:57:40.628382 15067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:40.635929 15067 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:40.637087 15067 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:40.644644 15067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce60fbb8471d44eaa0e1369a8c6386ca: Generated new TSK 0
I20260811 02:57:40.645372 15067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:40.677754 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46437
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:40.787329 15084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:40.787582 15084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:40.787623 15084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:40.791543 15084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:40.791599 15084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:40.791667 15084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:40.796892 15084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:40.797955 15084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:40.798981 15084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:40.808281 15084 server_base.cc:1034] running on GCE node
W20260811 02:57:40.808256 15101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:40.808243 15095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:40.808245 15097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:40.808792 15084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:40.809435 15084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:40.810602 15084 hybrid_clock.cc:648] HybridClock initialized: now 1786417060810582 us; error 29 us; skew 500 ppm
Aug 11 02:57:40 dist-test-slave-gjw7 krb5kdc[14876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:40.813130 15084 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:40.814003 15084 webserver.cc:460] Webserver started at http://127.8.203.65:38951/ using document root <none> and password file <none>
I20260811 02:57:40.814530 15084 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:40.814594 15084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:40.814792 15084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:40.816756 15084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "01275ca4ac2744f8aebd72107856bdb6"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-gjw7"
server_key: "dc3381f1d5e81432137843a34163b5c7"
server_key_iv: "288f0eece087bebd62bd97c611e9864f"
server_key_version: "encryptionkey@0"
I20260811 02:57:40.817358 15084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "01275ca4ac2744f8aebd72107856bdb6"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-gjw7"
server_key: "dc3381f1d5e81432137843a34163b5c7"
server_key_iv: "288f0eece087bebd62bd97c611e9864f"
server_key_version: "encryptionkey@0"
I20260811 02:57:40.821002 15084 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:57:40.823318 15115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:40.824477 15084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:40.824599 15084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "01275ca4ac2744f8aebd72107856bdb6"
format_stamp: "Formatted at 2026-08-11 02:57:40 on dist-test-slave-gjw7"
server_key: "dc3381f1d5e81432137843a34163b5c7"
server_key_iv: "288f0eece087bebd62bd97c611e9864f"
server_key_version: "encryptionkey@0"
I20260811 02:57:40.824688 15084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:40.848850 15084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:40.851581 15084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:40.851751 15084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:40.852310 15084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:40.853127 15084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:40.853174 15084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:40.853228 15084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:40.853256 15084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:40.861366 15084 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:43551
I20260811 02:57:40.861397 15358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:43551 every 8 connection(s)
I20260811 02:57:40.862205 15084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:40.863371 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15084
I20260811 02:57:40.863492 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:40.863807 9005 external_mini_cluster.cc:1428] Setting key f619abdbffc23e18395269896b499fed
Aug 11 02:57:40 dist-test-slave-gjw7 krb5kdc[14876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417060, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:57:40.876598 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:40.863949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:38505 (local address 127.8.203.126:46437)
0811 02:57:40.864074 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:40.864078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:40.865110 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 02:57:40.866977 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:40.866980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:40.867026 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:40.867085 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:40.868376 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:40.869121 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:40.870252 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:40.870619 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:40.870726 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:40.873235 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:40.873238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:40.873240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:40.873266 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:40.874634 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:40.875126 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:40.875128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:40.875130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:40.875209 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:40.875536 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:40.875539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:40.875541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:40.875721 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:40.875808 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:40.876301 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 02:57:40.876431 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 02:57:40.877449 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:40.864420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46437 (local address 127.8.203.65:38505)
0811 02:57:40.864970 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:40.865026 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:40.866734 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:40.867284 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:40.867293 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:40.867734 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:40.868225 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:40.868239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:40.869225 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:40.869227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:40.870133 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:40.870142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:40.870759 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:40.870762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:40.870911 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:40.871464 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:40.871481 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:40.873069 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:40.874741 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:40.874746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:40.874759 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:40.875019 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:40.875293 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:40.875297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:40.875299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:40.875444 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:40.875822 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:40.875828 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:40.876074 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 02:57:40.876239 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 02:57:40.876569 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260811 02:57:40.878716 15360 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46437
I20260811 02:57:40.878949 15360 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:40.879443 15360 heartbeater.cc:511] Master 127.8.203.126:46437 requested a full tablet report, sending...
I20260811 02:57:40.880880 14964 ts_manager.cc:194] Registered new tserver with Master: 01275ca4ac2744f8aebd72107856bdb6 (127.8.203.65:43551)
I20260811 02:57:40.882046 14964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:38505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:40.891258 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46437
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:41.000963 15372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:41.001190 15372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:41.001245 15372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:41.005102 15372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:41.005154 15372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:41.005229 15372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:41.010182 15372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:41.011101 15372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:41.012045 15372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:41.021199 15390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:41.021207 15381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:41.021209 15387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:41.021451 15372 server_base.cc:1034] running on GCE node
I20260811 02:57:41.021893 15372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:41.022454 15372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:41.023605 15372 hybrid_clock.cc:648] HybridClock initialized: now 1786417061023590 us; error 25 us; skew 500 ppm
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[14876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:41.026196 15372 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:41.026984 15372 webserver.cc:460] Webserver started at http://127.8.203.66:45991/ using document root <none> and password file <none>
I20260811 02:57:41.027489 15372 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:41.027552 15372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:41.027732 15372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:41.029716 15372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "1e9bf24eb29641d89251c83cd1cdfbea"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-gjw7"
server_key: "3550b620e7bf3a0b0d38497354976be3"
server_key_iv: "1655043b04714f6c85d19c705ac0f900"
server_key_version: "encryptionkey@0"
I20260811 02:57:41.030236 15372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "1e9bf24eb29641d89251c83cd1cdfbea"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-gjw7"
server_key: "3550b620e7bf3a0b0d38497354976be3"
server_key_iv: "1655043b04714f6c85d19c705ac0f900"
server_key_version: "encryptionkey@0"
I20260811 02:57:41.033694 15372 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:41.035894 15407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:41.037030 15372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:41.037140 15372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "1e9bf24eb29641d89251c83cd1cdfbea"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-gjw7"
server_key: "3550b620e7bf3a0b0d38497354976be3"
server_key_iv: "1655043b04714f6c85d19c705ac0f900"
server_key_version: "encryptionkey@0"
I20260811 02:57:41.037228 15372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:41.052632 15372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:41.055220 15372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:41.055390 15372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:41.055943 15372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:41.056716 15372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:41.056762 15372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:41.056814 15372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:41.056850 15372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:41.065251 15372 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:37667
I20260811 02:57:41.065303 15641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:37667 every 8 connection(s)
I20260811 02:57:41.066128 15372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:41.066881 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15372
I20260811 02:57:41.066978 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:41.067210 9005 external_mini_cluster.cc:1428] Setting key 1f7a9c0acd951021271263597ebd41c9
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[14876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
I20260811 02:57:41.081511 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:41.068744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:42543 (local address 127.8.203.126:46437)
0811 02:57:41.068881 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:41.068885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:41.069917 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 02:57:41.071432 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:41.071435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:41.071491 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:41.071573 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:41.072964 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.073633 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.075285 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.075506 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.075585 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:41.078309 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:41.078312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:41.078314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:41.078338 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:41.079542 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:41.080014 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:41.080017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:41.080018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:41.080094 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:41.080407 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:41.080410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:41.080412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:41.080592 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:41.080683 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:41.081243 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 02:57:41.081383 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:57:41.082573 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:41.069193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46437 (local address 127.8.203.66:42543)
0811 02:57:41.069744 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:41.069808 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:41.071228 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:41.071728 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:41.071735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:41.072134 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:41.072804 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:41.072824 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.073776 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.073780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:41.075168 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:41.075181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.075652 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.075657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:41.075869 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:41.076456 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:41.076476 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:41.078117 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:41.079650 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:41.079656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:41.079671 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:41.079918 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:41.080175 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:41.080178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:41.080180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:41.080311 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:41.080716 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:41.080723 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:41.080966 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 02:57:41.081175 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 02:57:41.081569 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":127,"threads_started":1}
I20260811 02:57:41.083655 15644 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46437
I20260811 02:57:41.083894 15644 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:41.084357 15644 heartbeater.cc:511] Master 127.8.203.126:46437 requested a full tablet report, sending...
I20260811 02:57:41.085413 14964 ts_manager.cc:194] Registered new tserver with Master: 1e9bf24eb29641d89251c83cd1cdfbea (127.8.203.66:37667)
I20260811 02:57:41.086164 14964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:42543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:41.095288 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46437
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:41.205945 15652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:41.206178 15652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:41.206218 15652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:41.210094 15652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:41.210152 15652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:41.210227 15652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:41.215268 15652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:41.216235 15652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:41.217182 15652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:41.226428 15671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:41.226428 15665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:41.226603 15652 server_base.cc:1034] running on GCE node
W20260811 02:57:41.226505 15667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:41.226985 15652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:41.227613 15652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:41.228792 15652 hybrid_clock.cc:648] HybridClock initialized: now 1786417061228761 us; error 42 us; skew 500 ppm
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[14876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:41.231299 15652 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:41.232086 15652 webserver.cc:460] Webserver started at http://127.8.203.67:35279/ using document root <none> and password file <none>
I20260811 02:57:41.232595 15652 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:41.232652 15652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:41.232839 15652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:41.234766 15652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "c244e0027cb6475cb18385695416ed87"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-gjw7"
server_key: "b02a07498e2ca14b87880ff117b31aff"
server_key_iv: "cb060229c426442fc89c1c44885f5534"
server_key_version: "encryptionkey@0"
I20260811 02:57:41.235281 15652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "c244e0027cb6475cb18385695416ed87"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-gjw7"
server_key: "b02a07498e2ca14b87880ff117b31aff"
server_key_iv: "cb060229c426442fc89c1c44885f5534"
server_key_version: "encryptionkey@0"
I20260811 02:57:41.238698 15652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 02:57:41.240856 15686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:41.242058 15652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:41.242173 15652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "c244e0027cb6475cb18385695416ed87"
format_stamp: "Formatted at 2026-08-11 02:57:41 on dist-test-slave-gjw7"
server_key: "b02a07498e2ca14b87880ff117b31aff"
server_key_iv: "cb060229c426442fc89c1c44885f5534"
server_key_version: "encryptionkey@0"
I20260811 02:57:41.242260 15652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:41.257072 15652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:41.259552 15652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:41.259711 15652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:41.260232 15652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:41.260998 15652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:41.261044 15652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:41.261092 15652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:41.261118 15652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:41.269050 15652 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:39295
I20260811 02:57:41.269083 15915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:39295 every 8 connection(s)
I20260811 02:57:41.269927 15652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:41.270987 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15652
I20260811 02:57:41.271081 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:41.271330 9005 external_mini_cluster.cc:1428] Setting key 9a002d63a4068b61ada225db3d9930d5
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[14876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:41.282518 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:41.271609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:44871 (local address 127.8.203.126:46437)
0811 02:57:41.271726 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:41.271730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:41.272642 (+ 912us) server_negotiation.cc:407] Connection header received
0811 02:57:41.274082 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:41.274086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:41.274142 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:41.274212 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:41.275327 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.275852 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.276951 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.277167 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.277253 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:41.279526 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:41.279529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:41.279531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:41.279554 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:41.280621 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:41.281067 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:41.281071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:41.281072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:41.281148 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:41.281504 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:41.281507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:41.281509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:41.281680 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:41.281767 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:41.282225 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 02:57:41.282368 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 02:57:41.283200 15924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:41.272018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46437 (local address 127.8.203.67:44871)
0811 02:57:41.272502 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:41.272553 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:41.273887 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:41.274400 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:41.274407 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:41.274740 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:41.275191 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:41.275205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.275976 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.275979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:41.276833 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:41.276845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.277331 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.277344 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:41.277502 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:41.278010 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:41.278026 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:41.279376 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:41.280723 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:41.280729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:41.280742 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:41.280975 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:41.281227 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:41.281230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:41.281232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:41.281414 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:41.281775 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:41.281782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:41.282002 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 02:57:41.282149 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 02:57:41.282453 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":133,"threads_started":1}
I20260811 02:57:41.284224 15921 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46437
I20260811 02:57:41.284437 15921 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:41.284904 15921 heartbeater.cc:511] Master 127.8.203.126:46437 requested a full tablet report, sending...
I20260811 02:57:41.285781 14964 ts_manager.cc:194] Registered new tserver with Master: c244e0027cb6475cb18385695416ed87 (127.8.203.67:39295)
I20260811 02:57:41.286280 14964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:44871
I20260811 02:57:41.296100 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[14876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[14876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417061, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:41.320447 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:41.313074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58868 (local address 127.8.203.126:46437)
0811 02:57:41.313222 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:41.313227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:41.313392 (+ 165us) server_negotiation.cc:407] Connection header received
0811 02:57:41.313516 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:41.313519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:41.313561 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:41.313617 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:41.314227 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.314738 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.315460 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:41.315652 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:41.315714 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:41.317889 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:41.317892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:41.317894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:41.317917 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:41.319043 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:41.319397 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:41.319399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:41.319400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:41.319495 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:41.319704 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:41.319706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:41.319707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:41.319910 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:41.320024 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:41.320192 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 02:57:41.320282 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 02:57:41.320811 14964 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58868
I20260811 02:57:41.321667 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15084
I20260811 02:57:41.348450 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15372
I20260811 02:57:41.359602 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15652
I20260811 02:57:41.367430 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 14906
2026-08-11T02:57:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2269 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[15950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[15950](info): set up 2 sockets
Aug 11 02:57:41 dist-test-slave-gjw7 krb5kdc[15950](info): commencing operation
krb5kdc: starting...
W20260811 02:57:42.593425 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:42 dist-test-slave-gjw7 krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:42.709767 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43747
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:42.819455 15977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:42.819716 15977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:42.819756 15977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:42.823704 15977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:42.823763 15977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:42.823781 15977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:42.823798 15977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:42.823812 15977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:42.828948 15977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:42.829978 15977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:42.830921 15977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:42.838954 15989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:42.838992 15987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:42.838969 15993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:42.839114 15977 server_base.cc:1034] running on GCE node
I20260811 02:57:42.840034 15977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:42.841104 15977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:42.842263 15977 hybrid_clock.cc:648] HybridClock initialized: now 1786417062842239 us; error 35 us; skew 500 ppm
Aug 11 02:57:42 dist-test-slave-gjw7 krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417062, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:42.844930 15977 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:42.845808 15977 webserver.cc:460] Webserver started at http://127.8.203.126:40993/ using document root <none> and password file <none>
I20260811 02:57:42.846335 15977 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:42.846398 15977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:42.846585 15977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:42.848574 15977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "24f4c25be40c4f3eb245461b05df6cd4"
format_stamp: "Formatted at 2026-08-11 02:57:42 on dist-test-slave-gjw7"
server_key: "c66ac480ad80982df635e1ffb3251dae"
server_key_iv: "acf8fea5244dc0eb19cc363fc75d3ff4"
server_key_version: "encryptionkey@0"
I20260811 02:57:42.849118 15977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "24f4c25be40c4f3eb245461b05df6cd4"
format_stamp: "Formatted at 2026-08-11 02:57:42 on dist-test-slave-gjw7"
server_key: "c66ac480ad80982df635e1ffb3251dae"
server_key_iv: "acf8fea5244dc0eb19cc363fc75d3ff4"
server_key_version: "encryptionkey@0"
I20260811 02:57:42.852763 15977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:57:42.855116 16012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:42.856336 15977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:42.856441 15977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "24f4c25be40c4f3eb245461b05df6cd4"
format_stamp: "Formatted at 2026-08-11 02:57:42 on dist-test-slave-gjw7"
server_key: "c66ac480ad80982df635e1ffb3251dae"
server_key_iv: "acf8fea5244dc0eb19cc363fc75d3ff4"
server_key_version: "encryptionkey@0"
I20260811 02:57:42.856537 15977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:42.868805 15977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:42.871587 15977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:42.871752 15977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:42.878816 15977 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:43747
I20260811 02:57:42.878835 16123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:43747 every 8 connection(s)
I20260811 02:57:42.879771 15977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:42.882557 16125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:42.885035 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15977
I20260811 02:57:42.885177 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:42.885540 9005 external_mini_cluster.cc:1428] Setting key ec40eeaa87aab207dc1fcbd5990f3784
I20260811 02:57:42.889204 16125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4: Bootstrap starting.
I20260811 02:57:42.891871 16125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:42.892695 16125 log.cc:824] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:57:42 dist-test-slave-gjw7 krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:42.894932 16125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4: No bootstrap required, opened a new log
I20260811 02:57:42.898644 16125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24f4c25be40c4f3eb245461b05df6cd4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43747 } }
I20260811 02:57:42.898933 16125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:42.899004 16125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f4c25be40c4f3eb245461b05df6cd4, State: Initialized, Role: FOLLOWER
I20260811 02:57:42.899492 16125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24f4c25be40c4f3eb245461b05df6cd4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43747 } }
I20260811 02:57:42.899637 16125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:42.899709 16125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:42.899770 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:42.886943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.8.203.126:43747)
0811 02:57:42.887329 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:42.887353 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:42.887389 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:57:42.888125 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:42.888143 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:42.888492 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:42.888766 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:42.889770 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:42.890743 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:42.891898 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:42.892386 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:42.892501 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:42.895235 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:42.895240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:42.895253 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:42.895279 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:42.896909 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:42.897382 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:42.897388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:42.897392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:42.897508 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:42.897774 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:42.897776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:42.897778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:42.898166 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:42.898386 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:42.898658 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 02:57:42.898935 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":97,"threads_started":1}
I20260811 02:57:42.899816 16125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:42.900990 16125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24f4c25be40c4f3eb245461b05df6cd4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43747 } }
I20260811 02:57:42.901356 16125 leader_election.cc:302] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24f4c25be40c4f3eb245461b05df6cd4; no voters:
I20260811 02:57:42.901705 16125 leader_election.cc:288] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:42.901808 16134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:42.902021 16134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [term 1 LEADER]: Becoming Leader. State: Replica: 24f4c25be40c4f3eb245461b05df6cd4, State: Running, Role: LEADER
I20260811 02:57:42.902397 16134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24f4c25be40c4f3eb245461b05df6cd4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43747 } }
I20260811 02:57:42.902959 16125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:42.903473 16140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24f4c25be40c4f3eb245461b05df6cd4. Latest consensus state: current_term: 1 leader_uuid: "24f4c25be40c4f3eb245461b05df6cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24f4c25be40c4f3eb245461b05df6cd4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43747 } } }
I20260811 02:57:42.903683 16140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:42.903949 16136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24f4c25be40c4f3eb245461b05df6cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24f4c25be40c4f3eb245461b05df6cd4" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43747 } } }
I20260811 02:57:42.904035 16136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:42.904304 16144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:42.907292 16144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:42.913008 16144 catalog_manager.cc:1347] Generated new cluster ID: d66668dda3564259861fea82f16156a9
I20260811 02:57:42.913079 16144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:42.922364 16144 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:42.923238 16144 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:42.934600 16144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24f4c25be40c4f3eb245461b05df6cd4: Generated new TSK 0
I20260811 02:57:42.935220 16144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:42.968017 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43747
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:43.080000 16174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:43.080247 16174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:43.080307 16174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:43.084247 16174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:43.084299 16174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:43.084375 16174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:43.089382 16174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:43.090333 16174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:43.091279 16174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:43.100626 16174 server_base.cc:1034] running on GCE node
W20260811 02:57:43.100576 16185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:43.100591 16184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:43.100564 16195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:43.101351 16174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:43.101992 16174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:43.103154 16174 hybrid_clock.cc:648] HybridClock initialized: now 1786417063103140 us; error 27 us; skew 500 ppm
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:43.106096 16174 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:43.106968 16174 webserver.cc:460] Webserver started at http://127.8.203.65:41193/ using document root <none> and password file <none>
I20260811 02:57:43.107519 16174 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:43.107591 16174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:43.107810 16174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:43.109899 16174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "9b285fac420b4e61a5b1aac54b592b60"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "f4fe2c85125e1a3f14026a6aef117a5d"
server_key_iv: "48810a7fc4cf2615ee700be4cb61b4d8"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.110461 16174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "9b285fac420b4e61a5b1aac54b592b60"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "f4fe2c85125e1a3f14026a6aef117a5d"
server_key_iv: "48810a7fc4cf2615ee700be4cb61b4d8"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.114241 16174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:57:43.116655 16216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.117859 16174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:43.117975 16174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "9b285fac420b4e61a5b1aac54b592b60"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "f4fe2c85125e1a3f14026a6aef117a5d"
server_key_iv: "48810a7fc4cf2615ee700be4cb61b4d8"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.118074 16174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:43.132007 16174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:43.134665 16174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:43.134833 16174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:43.135388 16174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:43.136194 16174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:43.136240 16174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.136292 16174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:43.136322 16174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.144443 16174 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:33553
I20260811 02:57:43.144465 16448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:33553 every 8 connection(s)
I20260811 02:57:43.145256 16174 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:43.154263 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 16174
I20260811 02:57:43.154379 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:43.154640 9005 external_mini_cluster.cc:1428] Setting key ded406af387430153e284040c53b5077
I20260811 02:57:43.158241 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.146946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:60993 (local address 127.8.203.126:43747)
0811 02:57:43.147081 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:43.147086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:43.147857 (+ 771us) server_negotiation.cc:407] Connection header received
0811 02:57:43.149198 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:43.149202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:43.149260 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:43.149356 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:43.150471 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.151006 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.152113 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.152342 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.152434 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.154778 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:43.154781 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:43.154783 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:43.154808 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:43.156283 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.156754 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.156758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.156760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.156843 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.157153 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.157156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.157158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.157378 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:43.157466 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:43.157944 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 02:57:43.158080 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:57:43.159034 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.147301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43747 (local address 127.8.203.65:60993)
0811 02:57:43.147719 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:43.147759 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:43.148993 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:43.149532 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:43.149540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:43.149865 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:43.150320 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:43.150333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.151128 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.151131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:43.151995 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:43.152005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.152482 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.152486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:43.152641 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.153146 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:43.153163 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:43.154617 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:43.156393 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:43.156399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:43.156413 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:43.156660 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:43.156928 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:43.156931 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:43.156933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:43.157066 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:43.157477 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:43.157484 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:43.157719 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 02:57:43.157887 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 02:57:43.158188 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":110,"threads_started":1}
I20260811 02:57:43.160115 16450 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43747
I20260811 02:57:43.160353 16450 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:43.160991 16450 heartbeater.cc:511] Master 127.8.203.126:43747 requested a full tablet report, sending...
I20260811 02:57:43.162634 16048 ts_manager.cc:194] Registered new tserver with Master: 9b285fac420b4e61a5b1aac54b592b60 (127.8.203.65:33553)
I20260811 02:57:43.163751 16048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:60993
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:43.181203 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43747
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:43.289438 16456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:43.289686 16456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:43.289743 16456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:43.293654 16456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:43.293704 16456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:43.293778 16456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:43.299098 16456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:43.300078 16456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:43.301054 16456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:43.310638 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:43.310704 16456 server_base.cc:1034] running on GCE node
W20260811 02:57:43.310667 16473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:43.310639 16466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:43.311264 16456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:43.311856 16456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:43.313010 16456 hybrid_clock.cc:648] HybridClock initialized: now 1786417063312994 us; error 27 us; skew 500 ppm
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:43.315567 16456 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:43.316394 16456 webserver.cc:460] Webserver started at http://127.8.203.66:39089/ using document root <none> and password file <none>
I20260811 02:57:43.316912 16456 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:43.316972 16456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:43.317169 16456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:43.319139 16456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "8d5cf8cf7a474d9f82a0b50de12ce5ed"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "d6ac2ba3084c02427afd25de7d9b438f"
server_key_iv: "7b514eb77a4f17b8d9c3e546c4fda6fc"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.319679 16456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "8d5cf8cf7a474d9f82a0b50de12ce5ed"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "d6ac2ba3084c02427afd25de7d9b438f"
server_key_iv: "7b514eb77a4f17b8d9c3e546c4fda6fc"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.323287 16456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:43.325529 16494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.326653 16456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:43.326753 16456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "8d5cf8cf7a474d9f82a0b50de12ce5ed"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "d6ac2ba3084c02427afd25de7d9b438f"
server_key_iv: "7b514eb77a4f17b8d9c3e546c4fda6fc"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.326844 16456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:43.340270 16456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:43.342764 16456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:43.342918 16456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:43.343441 16456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:43.344183 16456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:43.344228 16456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.344269 16456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:43.344298 16456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.352264 16456 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:46211
I20260811 02:57:43.352314 16695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:46211 every 8 connection(s)
I20260811 02:57:43.353060 16456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:43.356631 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 16456
I20260811 02:57:43.356737 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:43.357033 9005 external_mini_cluster.cc:1428] Setting key fc8601892266286850d70ff457b169a5
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:43.366928 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.354693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:33949 (local address 127.8.203.126:43747)
0811 02:57:43.354815 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:43.354819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:43.355591 (+ 772us) server_negotiation.cc:407] Connection header received
0811 02:57:43.357067 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:43.357070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:43.357131 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:43.357203 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:43.358509 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.359018 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.360510 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.360829 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.360943 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.363451 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:43.363455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:43.363457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:43.363480 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:43.364999 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.365510 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.365514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.365516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.365605 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.365889 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.365892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.365894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.366112 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:43.366208 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:43.366652 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 02:57:43.366768 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:57:43.367583 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.355037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43747 (local address 127.8.203.66:33949)
0811 02:57:43.355463 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:43.355502 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:43.356829 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:43.357420 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:43.357430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:43.357840 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:43.358363 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:43.358385 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.359146 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.359150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:43.360371 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:43.360384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.361010 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.361013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:43.361146 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.361652 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:43.361668 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:43.363283 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:43.365109 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:43.365114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:43.365125 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:43.365419 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:43.365683 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:43.365685 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:43.365687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:43.365804 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:43.366196 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:43.366200 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:43.366418 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 02:57:43.366558 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 02:57:43.366786 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 02:57:43.368698 16697 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43747
I20260811 02:57:43.368916 16697 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:43.369400 16697 heartbeater.cc:511] Master 127.8.203.126:43747 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
I20260811 02:57:43.370462 16048 ts_manager.cc:194] Registered new tserver with Master: 8d5cf8cf7a474d9f82a0b50de12ce5ed (127.8.203.66:46211)
I20260811 02:57:43.371140 16048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:33949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:43.383937 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43747
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:43.492388 16706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:43.492621 16706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:43.492661 16706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:43.496601 16706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:43.496649 16706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:43.496716 16706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:43.501843 16706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:43.502792 16706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:43.503719 16706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:43.512959 16720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:43.512995 16706 server_base.cc:1034] running on GCE node
W20260811 02:57:43.512946 16714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:43.512934 16716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:43.513540 16706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:43.514082 16706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:43.515233 16706 hybrid_clock.cc:648] HybridClock initialized: now 1786417063515215 us; error 30 us; skew 500 ppm
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:43.517766 16706 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:43.518560 16706 webserver.cc:460] Webserver started at http://127.8.203.67:43467/ using document root <none> and password file <none>
I20260811 02:57:43.519084 16706 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:43.519141 16706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:43.519304 16706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:43.521219 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "f8b9cdc664f8431ea2905fa7c8cb0f6b"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "c673a69ee64ab5f5d3fe14ae903a0236"
server_key_iv: "f8ff6769f11db37ac24ba81440b4d962"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.521780 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "f8b9cdc664f8431ea2905fa7c8cb0f6b"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "c673a69ee64ab5f5d3fe14ae903a0236"
server_key_iv: "f8ff6769f11db37ac24ba81440b4d962"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.525205 16706 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:57:43.527483 16730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.528577 16706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:43.528681 16706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "f8b9cdc664f8431ea2905fa7c8cb0f6b"
format_stamp: "Formatted at 2026-08-11 02:57:43 on dist-test-slave-gjw7"
server_key: "c673a69ee64ab5f5d3fe14ae903a0236"
server_key_iv: "f8ff6769f11db37ac24ba81440b4d962"
server_key_version: "encryptionkey@0"
I20260811 02:57:43.528779 16706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:43.540510 16706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:43.543387 16706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:43.543543 16706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:43.544070 16706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:43.544817 16706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:43.544862 16706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.544924 16706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:43.544950 16706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:43.552807 16706 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:36511
I20260811 02:57:43.552862 16954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:36511 every 8 connection(s)
I20260811 02:57:43.553647 16706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:43.559332 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 16706
I20260811 02:57:43.559451 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:43.559693 9005 external_mini_cluster.cc:1428] Setting key ec598cb4cc609fdff9d43e84ba10281c
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:43.566258 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.555344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:52107 (local address 127.8.203.126:43747)
0811 02:57:43.555484 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:43.555488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:43.556263 (+ 775us) server_negotiation.cc:407] Connection header received
0811 02:57:43.557599 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:43.557603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:43.557650 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:43.557718 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:43.558756 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.559396 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.560705 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.561002 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.561103 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.563352 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:43.563355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:43.563357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:43.563382 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:43.564646 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.565040 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.565042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.565043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.565107 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.565391 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.565393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.565394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.565564 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:43.565635 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:43.566048 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 02:57:43.566140 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:57:43.567020 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.555705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43747 (local address 127.8.203.67:52107)
0811 02:57:43.556123 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:43.556165 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:43.557416 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:43.557872 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:43.557878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:43.558204 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:43.558649 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:43.558662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.559531 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.559534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:43.560591 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:43.560603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.561143 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.561146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:43.561317 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.561877 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:43.561892 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:43.563210 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:43.564728 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:43.564732 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:43.564743 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:43.564971 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:43.565166 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:43.565168 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:43.565170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:43.565285 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:43.565638 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:43.565642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:43.565857 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 02:57:43.566000 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 02:57:43.566230 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
I20260811 02:57:43.568054 16957 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43747
I20260811 02:57:43.568250 16957 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:43.568670 16957 heartbeater.cc:511] Master 127.8.203.126:43747 requested a full tablet report, sending...
I20260811 02:57:43.569600 16048 ts_manager.cc:194] Registered new tserver with Master: f8b9cdc664f8431ea2905fa7c8cb0f6b (127.8.203.67:36511)
I20260811 02:57:43.570117 16048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:52107
I20260811 02:57:43.573275 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:43.598270 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.589784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.8.203.65:33553)
0811 02:57:43.590083 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:43.590092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:43.590116 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:57:43.590164 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:43.590168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:43.590344 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:43.590455 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:43.591048 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.591791 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.592579 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.592938 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.593014 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.595179 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:43.595188 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:43.595200 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:43.595222 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:43.596510 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.596908 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.596913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.596914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.596992 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.597235 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.597238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.597240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.597507 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:43.597707 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:43.597922 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 02:57:43.598070 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":95,"threads_started":1}
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[15950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:43.607861 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:43.600790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.8.203.126:43747)
0811 02:57:43.600934 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:43.600937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:43.600951 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:43.601036 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:43.601038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:43.601076 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:43.601137 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:43.601670 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.602166 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.602814 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:43.603054 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:43.603126 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:43.605363 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:43.605369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:43.605379 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:43.605409 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:43.606659 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.606995 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.606997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.606999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.607062 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:43.607233 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:43.607235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:43.607236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:43.607431 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:43.607527 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:43.607664 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 02:57:43.607749 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 02:57:43.608220 16048 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47484
I20260811 02:57:43.608729 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 16174
I20260811 02:57:43.620580 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 16456
I20260811 02:57:43.641440 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 16706
I20260811 02:57:43.669651 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15977
2026-08-11T02:57:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2306 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[16979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[16979](info): set up 2 sockets
Aug 11 02:57:43 dist-test-slave-gjw7 krb5kdc[16979](info): commencing operation
krb5kdc: starting...
W20260811 02:57:44.878307 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:44 dist-test-slave-gjw7 krb5kdc[16979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:44.993657 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:35025
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:35025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:45.102064 17007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:45.102294 17007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:45.102332 17007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:45.106175 17007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:45.106221 17007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:45.106240 17007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:45.106261 17007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:45.106274 17007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:45.111213 17007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:35025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:35025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:45.112107 17007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:45.113008 17007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:45.120719 17013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:45.120846 17007 server_base.cc:1034] running on GCE node
W20260811 02:57:45.120713 17014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:45.120728 17020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:45.121577 17007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:45.122604 17007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:45.123768 17007 hybrid_clock.cc:648] HybridClock initialized: now 1786417065123753 us; error 28 us; skew 500 ppm
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:45.132997 17007 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:45.133711 17007 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:45.133773 17007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:45.133939 17007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:45.135900 17007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "c80ff110ab2443c6bb078d7075e9e5e9"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "9f4d28bf29c5bed5dd88a89822968e84"
server_key_iv: "a6201a305e665198321cd7d25877f4de"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.136436 17007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "c80ff110ab2443c6bb078d7075e9e5e9"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "9f4d28bf29c5bed5dd88a89822968e84"
server_key_iv: "a6201a305e665198321cd7d25877f4de"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.139883 17007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:45.142108 17033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.143275 17007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 02:57:45.143386 17007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "c80ff110ab2443c6bb078d7075e9e5e9"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "9f4d28bf29c5bed5dd88a89822968e84"
server_key_iv: "a6201a305e665198321cd7d25877f4de"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.143481 17007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:45.153257 17007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:45.156117 17007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:45.156271 17007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:45.162727 17007 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:35025
I20260811 02:57:45.162976 17154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:35025 every 8 connection(s)
I20260811 02:57:45.163314 17007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:45.165985 17155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:45.168476 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17007
I20260811 02:57:45.168581 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:45.168829 9005 external_mini_cluster.cc:1428] Setting key b567029503ef94fff7a282b208bca4ae
I20260811 02:57:45.171655 17155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9: Bootstrap starting.
I20260811 02:57:45.174249 17155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:45.175096 17155 log.cc:824] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:45.177335 17155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9: No bootstrap required, opened a new log
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:45.180195 17155 raft_consensus.cc:348] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35025 } }
I20260811 02:57:45.180428 17155 raft_consensus.cc:374] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:45.180521 17155 raft_consensus.cc:729] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80ff110ab2443c6bb078d7075e9e5e9, State: Initialized, Role: FOLLOWER
I20260811 02:57:45.180991 17155 consensus_queue.cc:260] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35025 } }
I20260811 02:57:45.181173 17155 raft_consensus.cc:388] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:45.181244 17155 raft_consensus.cc:482] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:45.181365 17155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:45.182432 17155 raft_consensus.cc:504] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35025 } }
I20260811 02:57:45.182787 17155 leader_election.cc:302] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c80ff110ab2443c6bb078d7075e9e5e9; no voters:
I20260811 02:57:45.183104 17155 leader_election.cc:288] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:45.183192 17163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:45.183260 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.170113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.8.203.126:35025)
0811 02:57:45.170489 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.170505 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.170540 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:57:45.171141 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.171158 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.171483 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.171778 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:45.172639 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.173663 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.174830 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.175329 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.175473 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.178032 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:45.178037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:45.178050 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:45.178077 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:45.180108 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.180524 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.180532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.180538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.180692 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.180983 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.180986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.180988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.181574 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:45.181793 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.181985 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.182299 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":97,"threads_started":1}
I20260811 02:57:45.183413 17163 raft_consensus.cc:686] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [term 1 LEADER]: Becoming Leader. State: Replica: c80ff110ab2443c6bb078d7075e9e5e9, State: Running, Role: LEADER
I20260811 02:57:45.183728 17163 consensus_queue.cc:237] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35025 } }
I20260811 02:57:45.184258 17155 sys_catalog.cc:564] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:45.185534 17166 sys_catalog.cc:455] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35025 } } }
I20260811 02:57:45.185652 17166 sys_catalog.cc:458] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:45.185948 17167 sys_catalog.cc:455] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c80ff110ab2443c6bb078d7075e9e5e9. Latest consensus state: current_term: 1 leader_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c80ff110ab2443c6bb078d7075e9e5e9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35025 } } }
I20260811 02:57:45.186030 17183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:45.186045 17167 sys_catalog.cc:458] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:45.189051 17183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:45.194528 17183 catalog_manager.cc:1347] Generated new cluster ID: c6757661b87e4b54895ecad5fb46bf56
I20260811 02:57:45.194582 17183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:45.216784 17183 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:45.217815 17183 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:45.225337 17183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9: Generated new TSK 0
I20260811 02:57:45.225991 17183 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:45.255951 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:35025
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:45.366760 17202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:45.366989 17202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:45.367035 17202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:45.371160 17202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:45.371213 17202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:45.371302 17202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:45.376480 17202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:35025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:45.377436 17202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:45.378387 17202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:45.387241 17209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:45.387241 17218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:45.387394 17202 server_base.cc:1034] running on GCE node
W20260811 02:57:45.387517 17211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:45.387895 17202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:45.388473 17202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:45.389623 17202 hybrid_clock.cc:648] HybridClock initialized: now 1786417065389601 us; error 30 us; skew 500 ppm
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:45.392206 17202 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:45.392882 17202 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:45.392949 17202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:45.393149 17202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:45.395121 17202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "5c8feace99d142f3b4da1af8070f8fb2"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "f6792083dbe6acd43f48999a7007d447"
server_key_iv: "cfa9bfcdc4cf65ac1f9676fb03f7d384"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.395650 17202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "5c8feace99d142f3b4da1af8070f8fb2"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "f6792083dbe6acd43f48999a7007d447"
server_key_iv: "cfa9bfcdc4cf65ac1f9676fb03f7d384"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.399142 17202 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:45.401413 17228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.402634 17202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:45.402740 17202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "5c8feace99d142f3b4da1af8070f8fb2"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "f6792083dbe6acd43f48999a7007d447"
server_key_iv: "cfa9bfcdc4cf65ac1f9676fb03f7d384"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.402842 17202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:45.413941 17202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:45.416396 17202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:45.416553 17202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:45.417071 17202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:45.417872 17202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:45.417919 17202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.417969 17202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:45.417999 17202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.426172 17202 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:45739
I20260811 02:57:45.426187 17451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:45739 every 8 connection(s)
I20260811 02:57:45.426682 17202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:45.432659 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17202
I20260811 02:57:45.432768 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:45.433022 9005 external_mini_cluster.cc:1428] Setting key dc530aa9f1cc86fe1562b3b05a2dfe6d
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:45.439257 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.428209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:40061 (local address 127.8.203.126:35025)
0811 02:57:45.428351 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.428356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.429133 (+ 777us) server_negotiation.cc:407] Connection header received
0811 02:57:45.430365 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.430368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.430416 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.430488 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:45.431511 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.432037 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.433110 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.433382 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.433488 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.435953 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:45.435958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:45.435960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:45.435988 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:45.437255 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.437813 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.437817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.437818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.437923 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.438218 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.438221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.438222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.438426 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:45.438514 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.438959 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.439099 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:57:45.439899 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.428580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:35025 (local address 127.8.203.65:40061)
0811 02:57:45.428995 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:45.429033 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:45.430176 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:45.430628 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:45.430634 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:45.430954 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:45.431402 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.431416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.432121 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.432123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.432998 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.433008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.433527 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.433532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.433723 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.434265 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:45.434315 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:45.435769 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:45.437382 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:45.437388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:45.437402 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:45.437720 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:45.438013 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:45.438015 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:45.438016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:45.438139 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:45.438513 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:45.438518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:45.438720 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 02:57:45.438877 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 02:57:45.439106 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":113,"threads_started":1}
I20260811 02:57:45.440996 17454 heartbeater.cc:347] Connected to a master server at 127.8.203.126:35025
I20260811 02:57:45.441236 17454 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:45.441740 17454 heartbeater.cc:511] Master 127.8.203.126:35025 requested a full tablet report, sending...
I20260811 02:57:45.443205 17070 ts_manager.cc:194] Registered new tserver with Master: 5c8feace99d142f3b4da1af8070f8fb2 (127.8.203.65:45739)
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:57:45.444358 17070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:40061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:45.458232 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:35025
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:45.564949 17460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:45.565163 17460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:45.565202 17460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:45.568997 17460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:45.569041 17460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:45.569094 17460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:45.573983 17460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:35025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:45.574855 17460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:45.575753 17460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:45.584589 17470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:45.584604 17471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:45.584604 17479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:45.585354 17460 server_base.cc:1034] running on GCE node
I20260811 02:57:45.585728 17460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:45.586331 17460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:45.587491 17460 hybrid_clock.cc:648] HybridClock initialized: now 1786417065587477 us; error 27 us; skew 500 ppm
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:45.589965 17460 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:45.590644 17460 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:45.590705 17460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:45.590900 17460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:45.592839 17460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "34dadf51b9b243fd8dc8aa1642bd7426"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "81cc02b9f63cfccb47c7521c2523411b"
server_key_iv: "e7e6a78301e893b284f143e26bb7f86c"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.593402 17460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "34dadf51b9b243fd8dc8aa1642bd7426"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "81cc02b9f63cfccb47c7521c2523411b"
server_key_iv: "e7e6a78301e893b284f143e26bb7f86c"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.596802 17460 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:57:45.599133 17496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.600262 17460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:45.600356 17460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "34dadf51b9b243fd8dc8aa1642bd7426"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "81cc02b9f63cfccb47c7521c2523411b"
server_key_iv: "e7e6a78301e893b284f143e26bb7f86c"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.600457 17460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:45.625916 17460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:45.628484 17460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:45.628643 17460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:45.629195 17460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:45.630017 17460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:45.630062 17460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.630105 17460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:45.630134 17460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.638351 17460 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:42239
I20260811 02:57:45.638371 17731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:42239 every 8 connection(s)
I20260811 02:57:45.638916 17460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:45.643476 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17460
I20260811 02:57:45.643580 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:45.643848 9005 external_mini_cluster.cc:1428] Setting key abe62893dc16d6e16ded78360f096b31
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:45.651451 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.640504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:47185 (local address 127.8.203.126:35025)
0811 02:57:45.640619 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.640623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.641442 (+ 819us) server_negotiation.cc:407] Connection header received
0811 02:57:45.642709 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.642712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.642770 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.642846 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:45.644026 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.644568 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.645683 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.645913 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.645999 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.648259 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:45.648262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:45.648265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:45.648288 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:45.649571 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.650050 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.650053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.650054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.650130 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.650431 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.650434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.650436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.650609 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:45.650692 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.651165 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.651283 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 02:57:45.652258 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.640826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:35025 (local address 127.8.203.66:47185)
0811 02:57:45.641270 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:45.641331 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:45.642497 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:45.643026 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:45.643033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:45.643361 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:45.643877 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.643897 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.644687 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.644690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.645570 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.645579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.646047 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.646051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.646205 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.646743 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:45.646759 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:45.648108 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:45.649674 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:45.649681 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:45.649695 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:45.649959 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:45.650210 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:45.650213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:45.650215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:45.650344 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:45.650705 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:45.650711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:45.650943 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 02:57:45.651103 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 02:57:45.651392 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 02:57:45.653517 17734 heartbeater.cc:347] Connected to a master server at 127.8.203.126:35025
I20260811 02:57:45.653723 17734 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:45.654127 17734 heartbeater.cc:511] Master 127.8.203.126:35025 requested a full tablet report, sending...
I20260811 02:57:45.655069 17070 ts_manager.cc:194] Registered new tserver with Master: 34dadf51b9b243fd8dc8aa1642bd7426 (127.8.203.66:42239)
I20260811 02:57:45.655613 17070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:47185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:45.669142 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:35025
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:45.777738 17743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:45.777969 17743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:45.778028 17743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:45.781906 17743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:45.781955 17743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:45.782032 17743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:45.787149 17743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:35025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:45.788055 17743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:45.788964 17743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:45.797749 17759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:45.797747 17764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:45.797760 17757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:45.798118 17743 server_base.cc:1034] running on GCE node
I20260811 02:57:45.798504 17743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:45.799072 17743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:45.800271 17743 hybrid_clock.cc:648] HybridClock initialized: now 1786417065800253 us; error 29 us; skew 500 ppm
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:45.803078 17743 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:45.803804 17743 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:45.803875 17743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:45.804100 17743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:45.806143 17743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "b253d5f5a6d843cc8924b185fed0ede3"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "3298a99cc1af7296dcc0db979ad0bdcf"
server_key_iv: "1f54267fac569556b027d701538792d6"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.806676 17743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "b253d5f5a6d843cc8924b185fed0ede3"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "3298a99cc1af7296dcc0db979ad0bdcf"
server_key_iv: "1f54267fac569556b027d701538792d6"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.810173 17743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:45.812440 17774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.813611 17743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:45.813714 17743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "b253d5f5a6d843cc8924b185fed0ede3"
format_stamp: "Formatted at 2026-08-11 02:57:45 on dist-test-slave-gjw7"
server_key: "3298a99cc1af7296dcc0db979ad0bdcf"
server_key_iv: "1f54267fac569556b027d701538792d6"
server_key_version: "encryptionkey@0"
I20260811 02:57:45.813800 17743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:45.826920 17743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:45.829475 17743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:45.829629 17743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:45.830145 17743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:45.830907 17743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:45.830952 17743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.831004 17743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:45.831034 17743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:45.839049 17743 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:39631
I20260811 02:57:45.839087 18001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:39631 every 8 connection(s)
I20260811 02:57:45.839586 17743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:45.844415 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17743
I20260811 02:57:45.844532 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:45.844870 9005 external_mini_cluster.cc:1428] Setting key 18b283b6eb8558bcf6eaf1bdb0fa97e5
Aug 11 02:57:45 dist-test-slave-gjw7 krb5kdc[16979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:45.852540 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.841160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:51639 (local address 127.8.203.126:35025)
0811 02:57:45.841308 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.841313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.842231 (+ 918us) server_negotiation.cc:407] Connection header received
0811 02:57:45.843465 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.843468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.843516 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.843586 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:45.844959 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.845654 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.847072 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.847347 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.847426 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.849650 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:45.849653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:45.849655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:45.849680 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:45.850888 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.851347 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.851350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.851351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.851420 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.851658 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.851660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.851661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.851817 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:45.851897 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.852331 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.852425 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:57:45.853322 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.841551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:35025 (local address 127.8.203.67:51639)
0811 02:57:45.842075 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:45.842120 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:45.843284 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:45.843748 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:45.843755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:45.844089 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:45.844757 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.844780 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.845797 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.845800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.846947 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.846959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.847468 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.847471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.847613 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.848110 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:45.848126 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:45.849503 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:45.850997 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:45.851002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:45.851013 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:45.851270 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:45.851478 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:45.851480 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:45.851481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:45.851598 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:45.851913 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:45.851918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:45.852132 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 02:57:45.852299 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:57:45.852525 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":123,"threads_started":1}
I20260811 02:57:45.854308 18006 heartbeater.cc:347] Connected to a master server at 127.8.203.126:35025
I20260811 02:57:45.854498 18006 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:45.854879 18006 heartbeater.cc:511] Master 127.8.203.126:35025 requested a full tablet report, sending...
I20260811 02:57:45.855774 17070 ts_manager.cc:194] Registered new tserver with Master: b253d5f5a6d843cc8924b185fed0ede3 (127.8.203.67:39631)
I20260811 02:57:45.856303 17070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:51639
I20260811 02:57:45.858366 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:57:45.866461 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.860110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.8.203.126:35025)
0811 02:57:45.860201 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.860206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.860403 (+ 197us) server_negotiation.cc:407] Connection header received
0811 02:57:45.860506 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.860509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.860546 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.860606 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:45.861243 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.861754 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.862540 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.862744 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.862835 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.863957 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:45.863960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:45.863962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:45.863983 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:45.864952 (+ 969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.865348 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.865352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.865353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.865429 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:45.865666 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:45.865669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:45.865671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:45.865874 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:45.865991 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.866218 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.866324 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 02:57:45.868834 17070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:45.871711 17070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:45.886955 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.879844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:39631 (local address 127.0.0.1:33562)
0811 02:57:45.880249 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:45.880317 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:45.880483 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:45.882031 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:45.882038 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:45.882063 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:45.882198 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.882204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.883606 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.883610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.885277 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.885329 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.886420 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.886424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.886528 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.886564 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 02:57:45.886674 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 02:57:45.886796 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1,"thread_start_us":78,"threads_started":1}
I20260811 02:57:45.886960 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.879983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33562 (local address 127.8.203.67:39631)
0811 02:57:45.881544 (+ 1561us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.881557 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.881582 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:57:45.881660 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.881665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.881872 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.882027 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:45.882329 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.883445 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.885551 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.886306 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.886458 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.886617 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.886702 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.886796 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1454,"thread_start_us":144,"threads_started":1}
I20260811 02:57:45.887913 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.880885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:42239 (local address 127.0.0.1:48368)
0811 02:57:45.881159 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:45.881195 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:45.881354 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:45.883245 (+ 1891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:45.883248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:45.883265 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:45.883365 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.883371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.884810 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.884814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.885976 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.885987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.887466 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.887470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.887559 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.887620 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 02:57:45.887693 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:57:45.887764 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":27,"thread_start_us":98,"threads_started":1}
I20260811 02:57:45.888121 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.882305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.8.203.66:42239)
0811 02:57:45.882623 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.882634 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.882656 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:57:45.882733 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.882738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.882944 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.883082 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:45.883492 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.884665 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.886123 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.886964 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.887249 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.887408 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.887747 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.887854 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":108,"threads_started":1}
I20260811 02:57:45.888247 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.880876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.8.203.65:45739)
0811 02:57:45.881771 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:45.881783 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:45.882862 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 02:57:45.882942 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:45.882946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:45.883122 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:45.883251 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:45.883637 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.884731 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.886438 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.887294 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.887388 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.887516 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:45.887984 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 02:57:45.888095 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":145,"threads_started":1}
I20260811 02:57:45.887913 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:45.881201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45739 (local address 127.0.0.1:46878)
0811 02:57:45.882672 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:45.882685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:45.882771 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:45.883424 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:45.883427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:45.883444 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:45.883517 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.883523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.884859 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.884862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.886265 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:45.886276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:45.887569 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:45.887572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:45.887669 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:45.887689 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:57:45.887745 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 02:57:45.887793 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1412,"mutex_wait_us":374,"thread_start_us":110,"threads_started":1}
I20260811 02:57:45.890144 17874 tablet_service.cc:1467] Processing CreateTablet for tablet 292647e56e3b4ae89a99ccb93966e26a (DEFAULT_TABLE table=test-table [id=6bc49ed00e7846799ee20226f2931ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:45.891038 17319 tablet_service.cc:1467] Processing CreateTablet for tablet 292647e56e3b4ae89a99ccb93966e26a (DEFAULT_TABLE table=test-table [id=6bc49ed00e7846799ee20226f2931ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:45.891355 17874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292647e56e3b4ae89a99ccb93966e26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:45.891888 17319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292647e56e3b4ae89a99ccb93966e26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:45.891841 17598 tablet_service.cc:1467] Processing CreateTablet for tablet 292647e56e3b4ae89a99ccb93966e26a (DEFAULT_TABLE table=test-table [id=6bc49ed00e7846799ee20226f2931ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:45.892833 17598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 292647e56e3b4ae89a99ccb93966e26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:45.897503 18034 tablet_bootstrap.cc:492] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: Bootstrap starting.
I20260811 02:57:45.899353 18039 tablet_bootstrap.cc:492] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Bootstrap starting.
I20260811 02:57:45.899906 18034 tablet_bootstrap.cc:654] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:45.900331 18037 tablet_bootstrap.cc:492] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Bootstrap starting.
I20260811 02:57:45.901121 18034 log.cc:824] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:45.901420 18039 tablet_bootstrap.cc:654] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:45.902530 18037 tablet_bootstrap.cc:654] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:45.902642 18039 log.cc:824] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:45.902958 18034 tablet_bootstrap.cc:492] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: No bootstrap required, opened a new log
I20260811 02:57:45.903136 18034 ts_tablet_manager.cc:1397] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:57:45.903716 18037 log.cc:824] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:45.904699 18039 tablet_bootstrap.cc:492] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: No bootstrap required, opened a new log
I20260811 02:57:45.904891 18039 ts_tablet_manager.cc:1397] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 02:57:45.905519 18037 tablet_bootstrap.cc:492] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: No bootstrap required, opened a new log
I20260811 02:57:45.905683 18037 ts_tablet_manager.cc:1397] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:45.906859 18034 raft_consensus.cc:348] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:45.907127 18034 raft_consensus.cc:374] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:45.907183 18034 raft_consensus.cc:729] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c8feace99d142f3b4da1af8070f8fb2, State: Initialized, Role: FOLLOWER
I20260811 02:57:45.907641 18034 consensus_queue.cc:260] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:45.908609 18034 ts_tablet_manager.cc:1428] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260811 02:57:45.908471 18039 raft_consensus.cc:348] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:45.908715 18039 raft_consensus.cc:374] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:45.908768 18039 raft_consensus.cc:729] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dadf51b9b243fd8dc8aa1642bd7426, State: Initialized, Role: FOLLOWER
I20260811 02:57:45.908670 18037 raft_consensus.cc:348] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:45.908931 18037 raft_consensus.cc:374] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:45.908980 18037 raft_consensus.cc:729] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b253d5f5a6d843cc8924b185fed0ede3, State: Initialized, Role: FOLLOWER
I20260811 02:57:45.908988 17454 heartbeater.cc:503] Master 127.8.203.126:35025 was elected leader, sending a full tablet report...
I20260811 02:57:45.909168 18039 consensus_queue.cc:260] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:45.909381 18037 consensus_queue.cc:260] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:45.909834 17734 heartbeater.cc:503] Master 127.8.203.126:35025 was elected leader, sending a full tablet report...
I20260811 02:57:45.910063 18039 ts_tablet_manager.cc:1428] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:57:45.911167 18037 ts_tablet_manager.cc:1428] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:45.911460 18006 heartbeater.cc:503] Master 127.8.203.126:35025 was elected leader, sending a full tablet report...
W20260811 02:57:45.927608 17456 tablet.cc:2369] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:46.028138 18043 raft_consensus.cc:482] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:46.028354 18043 raft_consensus.cc:504] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:46.029511 18043 leader_election.cc:288] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b253d5f5a6d843cc8924b185fed0ede3 (127.8.203.67:39631), 34dadf51b9b243fd8dc8aa1642bd7426 (127.8.203.66:42239)
I20260811 02:57:46.034592 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.029833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:45261 (local address 127.8.203.67:39631)
0811 02:57:46.029994 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.029999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.030197 (+ 198us) server_negotiation.cc:407] Connection header received
0811 02:57:46.030352 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.030355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.030399 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.030484 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:46.030850 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.031634 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.033372 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.033987 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.034108 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:46.034161 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.034397 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.034465 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 02:57:46.034592 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.029888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:52711 (local address 127.8.203.66:42239)
0811 02:57:46.030022 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.030026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.030181 (+ 155us) server_negotiation.cc:407] Connection header received
0811 02:57:46.030344 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.030347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.030399 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.030484 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:46.030856 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.031634 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.033371 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.033990 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.034108 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:46.034162 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.034397 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.034462 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:57:46.034634 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.029722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:42239 (local address 127.8.203.65:52711)
0811 02:57:46.030089 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.030103 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.030236 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.030508 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.030511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.030537 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:46.030748 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:46.030754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.031743 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.031746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:46.033226 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:46.033237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.034109 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.034113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:46.034255 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:46.034297 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.034383 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.034461 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":175,"spinlock_wait_cycles":35200,"thread_start_us":100,"threads_started":1}
I20260811 02:57:46.034636 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.029722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:39631 (local address 127.8.203.65:45261)
0811 02:57:46.030111 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.030123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.030236 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.030508 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.030511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.030537 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:46.030739 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:46.030747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.031743 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.031746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:46.033224 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:46.033237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.034109 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.034113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:46.034269 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:46.034303 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.034372 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.034468 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":201,"spinlock_wait_cycles":14336,"thread_start_us":109,"threads_started":1}
I20260811 02:57:46.035256 17642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292647e56e3b4ae89a99ccb93966e26a" candidate_uuid: "5c8feace99d142f3b4da1af8070f8fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" is_pre_election: true
I20260811 02:57:46.035549 17915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292647e56e3b4ae89a99ccb93966e26a" candidate_uuid: "5c8feace99d142f3b4da1af8070f8fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b253d5f5a6d843cc8924b185fed0ede3" is_pre_election: true
I20260811 02:57:46.035576 17642 raft_consensus.cc:2454] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c8feace99d142f3b4da1af8070f8fb2 in term 0.
I20260811 02:57:46.035812 17915 raft_consensus.cc:2454] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c8feace99d142f3b4da1af8070f8fb2 in term 0.
I20260811 02:57:46.036072 17233 leader_election.cc:302] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34dadf51b9b243fd8dc8aa1642bd7426, 5c8feace99d142f3b4da1af8070f8fb2; no voters:
I20260811 02:57:46.036356 18043 raft_consensus.cc:2781] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:46.036437 18043 raft_consensus.cc:482] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:46.036466 18043 raft_consensus.cc:3037] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:46.037501 18043 raft_consensus.cc:504] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:46.037828 18043 leader_election.cc:288] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [CANDIDATE]: Term 1 election: Requested vote from peers b253d5f5a6d843cc8924b185fed0ede3 (127.8.203.67:39631), 34dadf51b9b243fd8dc8aa1642bd7426 (127.8.203.66:42239)
I20260811 02:57:46.038136 17915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292647e56e3b4ae89a99ccb93966e26a" candidate_uuid: "5c8feace99d142f3b4da1af8070f8fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b253d5f5a6d843cc8924b185fed0ede3"
I20260811 02:57:46.038159 17642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "292647e56e3b4ae89a99ccb93966e26a" candidate_uuid: "5c8feace99d142f3b4da1af8070f8fb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34dadf51b9b243fd8dc8aa1642bd7426"
I20260811 02:57:46.038254 17642 raft_consensus.cc:3037] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:46.038250 17915 raft_consensus.cc:3037] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:46.039229 17642 raft_consensus.cc:2454] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c8feace99d142f3b4da1af8070f8fb2 in term 1.
I20260811 02:57:46.039316 17915 raft_consensus.cc:2454] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c8feace99d142f3b4da1af8070f8fb2 in term 1.
I20260811 02:57:46.039529 17233 leader_election.cc:302] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34dadf51b9b243fd8dc8aa1642bd7426, 5c8feace99d142f3b4da1af8070f8fb2; no voters:
I20260811 02:57:46.039687 18043 raft_consensus.cc:2781] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:46.039889 18043 raft_consensus.cc:686] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 1 LEADER]: Becoming Leader. State: Replica: 5c8feace99d142f3b4da1af8070f8fb2, State: Running, Role: LEADER
I20260811 02:57:46.040189 18043 consensus_queue.cc:237] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } }
I20260811 02:57:46.043323 17066 catalog_manager.cc:5495] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c8feace99d142f3b4da1af8070f8fb2 (127.8.203.65). New cstate: current_term: 1 leader_uuid: "5c8feace99d142f3b4da1af8070f8fb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c8feace99d142f3b4da1af8070f8fb2" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45739 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:46.080590 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.077588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46886 (local address 127.8.203.65:45739)
0811 02:57:46.077716 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.077719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.077733 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:46.077864 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.077866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.077920 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.078002 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:46.078196 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.078675 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.079433 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:46.079633 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:46.079697 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:46.079927 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:46.080316 (+ 389us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:46.080400 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.080441 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.080488 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:57:46.086320 17915 raft_consensus.cc:1261] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 1 FOLLOWER]: Refusing update from remote peer 5c8feace99d142f3b4da1af8070f8fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:46.086320 17642 raft_consensus.cc:1261] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 1 FOLLOWER]: Refusing update from remote peer 5c8feace99d142f3b4da1af8070f8fb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:46.087076 18043 consensus_queue.cc:1035] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34dadf51b9b243fd8dc8aa1642bd7426" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:46.087288 18049 consensus_queue.cc:1035] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b253d5f5a6d843cc8924b185fed0ede3" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:57:46.090387 18007 tablet.cc:2369] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:46.094961 18054 mvcc.cc:205] Tried to move back new op lower bound from 7317164302681944064 to 7317164302502989824. Current Snapshot: MvccSnapshot[applied={T|T < 7317164302681944064}]
I20260811 02:57:46.097470 18063 mvcc.cc:205] Tried to move back new op lower bound from 7317164302681944064 to 7317164302502989824. Current Snapshot: MvccSnapshot[applied={T|T < 7317164302681944064}]
I20260811 02:57:46.097713 18064 mvcc.cc:205] Tried to move back new op lower bound from 7317164302681944064 to 7317164302502989824. Current Snapshot: MvccSnapshot[applied={T|T < 7317164302681944064 or (T in {7317164302681944064})}]
I20260811 02:57:46.103019 17070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:46.103194 17070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:46.105458 17070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c80ff110ab2443c6bb078d7075e9e5e9: Sending DeleteTablet for 3 replicas of tablet 292647e56e3b4ae89a99ccb93966e26a
I20260811 02:57:46.106154 17874 tablet_service.cc:1514] Processing DeleteTablet for tablet 292647e56e3b4ae89a99ccb93966e26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:46 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:33562
I20260811 02:57:46.106331 17598 tablet_service.cc:1514] Processing DeleteTablet for tablet 292647e56e3b4ae89a99ccb93966e26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:46 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:48368
I20260811 02:57:46.106338 17319 tablet_service.cc:1514] Processing DeleteTablet for tablet 292647e56e3b4ae89a99ccb93966e26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:46 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:46878
I20260811 02:57:46.106842 18081 tablet_replica.cc:330] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: stopping tablet replica
I20260811 02:57:46.106851 18083 tablet_replica.cc:330] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: stopping tablet replica
I20260811 02:57:46.107154 18083 raft_consensus.cc:2229] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:46.107406 18086 tablet_replica.cc:330] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2: stopping tablet replica
I20260811 02:57:46.107451 18083 raft_consensus.cc:2258] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:46.107508 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17202
I20260811 02:57:46.107641 18086 raft_consensus.cc:2229] T 292647e56e3b4ae89a99ccb93966e26a P 5c8feace99d142f3b4da1af8070f8fb2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:46.109066 18083 ts_tablet_manager.cc:1905] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:46.112402 18083 ts_tablet_manager.cc:1918] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:46.112514 18083 log.cc:1196] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/292647e56e3b4ae89a99ccb93966e26a
I20260811 02:57:46.112855 18083 ts_tablet_manager.cc:1939] T 292647e56e3b4ae89a99ccb93966e26a P 34dadf51b9b243fd8dc8aa1642bd7426: Deleting consensus metadata
I20260811 02:57:46.107179 18081 raft_consensus.cc:2229] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:46.115026 18081 raft_consensus.cc:2258] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:46.115697 17043 catalog_manager.cc:4841] TS 34dadf51b9b243fd8dc8aa1642bd7426 (127.8.203.66:42239): tablet 292647e56e3b4ae89a99ccb93966e26a (table test-table [id=6bc49ed00e7846799ee20226f2931ff5]) successfully deleted
I20260811 02:57:46.116544 18081 ts_tablet_manager.cc:1905] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:46.119091 18081 ts_tablet_manager.cc:1918] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:46.119182 18081 log.cc:1196] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/292647e56e3b4ae89a99ccb93966e26a
I20260811 02:57:46.119483 18081 ts_tablet_manager.cc:1939] T 292647e56e3b4ae89a99ccb93966e26a P b253d5f5a6d843cc8924b185fed0ede3: Deleting consensus metadata
I20260811 02:57:46.120651 17034 catalog_manager.cc:4841] TS b253d5f5a6d843cc8924b185fed0ede3 (127.8.203.67:39631): tablet 292647e56e3b4ae89a99ccb93966e26a (table test-table [id=6bc49ed00e7846799ee20226f2931ff5]) successfully deleted
W20260811 02:57:46.122355 17038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:45739 (error 108)
I20260811 02:57:46.123086 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17460
I20260811 02:57:46.123328 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.123054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45739 (local address 127.0.0.1:46900)
0811 02:57:46.123185 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.123250 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 02:57:46.150625 17038 catalog_manager.cc:4584] TS 5c8feace99d142f3b4da1af8070f8fb2 (127.8.203.65:45739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 292647e56e3b4ae89a99ccb93966e26a: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45739: connect: Connection refused (error 111)
I20260811 02:57:46.155992 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17743
I20260811 02:57:46.172636 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17007
2026-08-11T02:57:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2500 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T02:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:46Z Disabled control of system clock
I20260811 02:57:46.208840 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:32997
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:32997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:57:46.316823 18104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:46.317050 18104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:46.317101 18104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:46.320940 18104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:46.320989 18104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:46.321010 18104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:46.321031 18104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:46.321051 18104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:46.325593 18104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:32997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:32997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:46.326476 18104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:46.327682 18104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:46.335491 18111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:46.335552 18112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:46.335637 18104 server_base.cc:1034] running on GCE node
W20260811 02:57:46.335554 18114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:46.336382 18104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:46.337363 18104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:46.338524 18104 hybrid_clock.cc:648] HybridClock initialized: now 1786417066338508 us; error 33 us; skew 500 ppm
I20260811 02:57:46.340155 18104 webserver.cc:460] Webserver started at http://127.8.203.126:41289/ using document root <none> and password file <none>
I20260811 02:57:46.340670 18104 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:46.340734 18104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:46.340926 18104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:46.342893 18104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "fe1859368c604bc89392b16c0f00e49d"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "e89f73587c5ab91ef6500fee66a722ba"
server_key_iv: "acc5435536a03ba5216445f987b0e8f0"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.343438 18104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "fe1859368c604bc89392b16c0f00e49d"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "e89f73587c5ab91ef6500fee66a722ba"
server_key_iv: "acc5435536a03ba5216445f987b0e8f0"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.346884 18104 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 02:57:46.349077 18126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.350241 18104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:46.350353 18104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "fe1859368c604bc89392b16c0f00e49d"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "e89f73587c5ab91ef6500fee66a722ba"
server_key_iv: "acc5435536a03ba5216445f987b0e8f0"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.350451 18104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:46.354889 18104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:46.355446 18104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:46.355602 18104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:46.361997 18104 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:32997
I20260811 02:57:46.362015 18227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:32997 every 8 connection(s)
I20260811 02:57:46.362831 18104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:46.363726 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18104
I20260811 02:57:46.363842 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:46.364099 9005 external_mini_cluster.cc:1428] Setting key c2b5597256709334dc7a25c44c8d0890
I20260811 02:57:46.365872 18228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:46.369796 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.365454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.8.203.126:32997)
0811 02:57:46.366454 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.366469 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.366503 (+ 34us) server_negotiation.cc:407] Connection header received
0811 02:57:46.367124 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.367140 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.367462 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.367731 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.368567 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.368572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.368583 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.368594 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.368647 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.368662 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.368767 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.368888 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.369112 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":523,"threads_started":1}
I20260811 02:57:46.371424 18228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d: Bootstrap starting.
I20260811 02:57:46.373507 18228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:46.374246 18228 log.cc:824] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:46.375967 18228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d: No bootstrap required, opened a new log
I20260811 02:57:46.379359 18228 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1859368c604bc89392b16c0f00e49d" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 32997 } }
I20260811 02:57:46.379565 18228 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:46.379623 18228 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe1859368c604bc89392b16c0f00e49d, State: Initialized, Role: FOLLOWER
I20260811 02:57:46.379992 18228 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1859368c604bc89392b16c0f00e49d" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 32997 } }
I20260811 02:57:46.380112 18228 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:46.380177 18228 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:46.380273 18228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:46.381060 18228 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1859368c604bc89392b16c0f00e49d" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 32997 } }
I20260811 02:57:46.381404 18228 leader_election.cc:302] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe1859368c604bc89392b16c0f00e49d; no voters:
I20260811 02:57:46.381686 18228 leader_election.cc:288] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:46.381767 18238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:46.381970 18238 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [term 1 LEADER]: Becoming Leader. State: Replica: fe1859368c604bc89392b16c0f00e49d, State: Running, Role: LEADER
I20260811 02:57:46.382269 18238 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1859368c604bc89392b16c0f00e49d" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 32997 } }
I20260811 02:57:46.382836 18228 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:46.383781 18239 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe1859368c604bc89392b16c0f00e49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1859368c604bc89392b16c0f00e49d" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 32997 } } }
I20260811 02:57:46.383929 18239 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:46.383785 18242 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe1859368c604bc89392b16c0f00e49d. Latest consensus state: current_term: 1 leader_uuid: "fe1859368c604bc89392b16c0f00e49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1859368c604bc89392b16c0f00e49d" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 32997 } } }
I20260811 02:57:46.384238 18242 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:46.384357 18258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:46.387241 18258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:46.392586 18258 catalog_manager.cc:1347] Generated new cluster ID: b3aa0da7ea614c8ca931f4ef0653f4f6
I20260811 02:57:46.392633 18258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:46.411270 18258 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:46.411542 18258 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:46.421195 18258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d: Generated new TSK 0
I20260811 02:57:46.421802 18258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:46.425637 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:32997
--builtin_ntp_servers=127.8.203.84:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:57:46.546329 18282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:46.546546 18282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:46.546581 18282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:46.550396 18282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:46.550436 18282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:46.550485 18282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:46.554994 18282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:46.555899 18282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:46.557185 18282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:46.566512 18294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:46.566462 18282 server_base.cc:1034] running on GCE node
W20260811 02:57:46.566453 18298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:46.566448 18292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:46.567060 18282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:46.567644 18282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:46.568799 18282 hybrid_clock.cc:648] HybridClock initialized: now 1786417066568782 us; error 27 us; skew 500 ppm
I20260811 02:57:46.570570 18282 webserver.cc:460] Webserver started at http://127.8.203.65:44799/ using document root <none> and password file <none>
I20260811 02:57:46.571118 18282 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:46.571180 18282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:46.571381 18282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:46.573546 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "1e518574da0947d096666f8c66cde604"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "8b92a84087456e15d3dfcb91f706f27d"
server_key_iv: "bd69d43a68854a90f7d5ab5b89a34743"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.574070 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "1e518574da0947d096666f8c66cde604"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "8b92a84087456e15d3dfcb91f706f27d"
server_key_iv: "bd69d43a68854a90f7d5ab5b89a34743"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.577633 18282 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:57:46.579883 18312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.581014 18282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:46.581130 18282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "1e518574da0947d096666f8c66cde604"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "8b92a84087456e15d3dfcb91f706f27d"
server_key_iv: "bd69d43a68854a90f7d5ab5b89a34743"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.581223 18282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:46.585935 18282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:46.586501 18282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:46.586655 18282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:46.587175 18282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:46.588042 18282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:46.588088 18282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.588136 18282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:46.588166 18282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.596413 18282 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:36653
I20260811 02:57:46.596441 18527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:36653 every 8 connection(s)
I20260811 02:57:46.597259 18282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:46.602871 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18282
I20260811 02:57:46.602970 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:46.603147 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.598961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:58919 (local address 127.8.203.126:32997)
0811 02:57:46.599087 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.599091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.600090 (+ 999us) server_negotiation.cc:407] Connection header received
0811 02:57:46.601509 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.601512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.601560 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.601635 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.602698 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.602701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.602703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.602715 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.602736 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.602744 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.602822 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.602967 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.603013 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:57:46.603230 9005 external_mini_cluster.cc:1428] Setting key a1b8826aad6f443ff9f5e1bbdd2cd857
I20260811 02:57:46.604104 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.599302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:32997 (local address 127.8.203.65:58919)
0811 02:57:46.599944 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.599982 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.601273 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.601815 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.601822 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.601917 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:46.602425 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:46.602427 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:46.602466 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:46.602532 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:46.602817 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:46.602826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:46.602832 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.602997 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.603227 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260811 02:57:46.605150 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:32997
--builtin_ntp_servers=127.8.203.84:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 02:57:46.605216 18528 heartbeater.cc:347] Connected to a master server at 127.8.203.126:32997
I20260811 02:57:46.605449 18528 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:46.605819 18528 heartbeater.cc:511] Master 127.8.203.126:32997 requested a full tablet report, sending...
I20260811 02:57:46.607312 18157 ts_manager.cc:194] Registered new tserver with Master: 1e518574da0947d096666f8c66cde604 (127.8.203.65:36653)
W20260811 02:57:46.715286 18543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:46.715507 18543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:46.715540 18543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:46.719386 18543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:46.719434 18543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:46.719484 18543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:46.724135 18543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:46.725080 18543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:46.726397 18543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:46.735636 18543 server_base.cc:1034] running on GCE node
W20260811 02:57:46.735596 18552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:46.735653 18550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:46.735615 18557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:46.736249 18543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:46.736848 18543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:46.737994 18543 hybrid_clock.cc:648] HybridClock initialized: now 1786417066737979 us; error 26 us; skew 500 ppm
I20260811 02:57:46.739737 18543 webserver.cc:460] Webserver started at http://127.8.203.66:46583/ using document root <none> and password file <none>
I20260811 02:57:46.740268 18543 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:46.740330 18543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:46.740547 18543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:46.742637 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "8825a306f0c64765891999d3baed45a2"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "4fb07d044800a2d88664b3b6119fc29e"
server_key_iv: "ac784364cae7130bc8faf5229d3f2308"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.743170 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "8825a306f0c64765891999d3baed45a2"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "4fb07d044800a2d88664b3b6119fc29e"
server_key_iv: "ac784364cae7130bc8faf5229d3f2308"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.746661 18543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:46.748894 18567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.750051 18543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:46.750152 18543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "8825a306f0c64765891999d3baed45a2"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "4fb07d044800a2d88664b3b6119fc29e"
server_key_iv: "ac784364cae7130bc8faf5229d3f2308"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.750257 18543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:46.754838 18543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:46.755404 18543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:46.755556 18543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:46.756088 18543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:46.756963 18543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:46.757006 18543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.757046 18543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:46.757077 18543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.765413 18543 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:34901
I20260811 02:57:46.765446 18779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:34901 every 8 connection(s)
I20260811 02:57:46.766228 18543 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:46.771811 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.767854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:60273 (local address 127.8.203.126:32997)
0811 02:57:46.767984 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.767988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.768948 (+ 960us) server_negotiation.cc:407] Connection header received
0811 02:57:46.770245 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.770248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.770297 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.770382 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.771385 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.771389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.771391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.771403 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.771422 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.771431 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.771506 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.771640 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.771684 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:57:46.772079 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18543
I20260811 02:57:46.772176 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:46.772428 9005 external_mini_cluster.cc:1428] Setting key 659a572e622a88f2ac4e999c3bb5e8b4
I20260811 02:57:46.772691 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.768188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:32997 (local address 127.8.203.66:60273)
0811 02:57:46.768798 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.768836 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.770033 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.770549 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.770556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.770641 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:46.771122 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:46.771123 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:46.771162 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:46.771228 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:46.771506 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:46.771515 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:46.771522 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.771671 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.771912 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":100,"threads_started":1}
I20260811 02:57:46.773752 18781 heartbeater.cc:347] Connected to a master server at 127.8.203.126:32997
I20260811 02:57:46.773965 18781 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:46.774277 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:32997
--builtin_ntp_servers=127.8.203.84:38533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 02:57:46.774333 18781 heartbeater.cc:511] Master 127.8.203.126:32997 requested a full tablet report, sending...
I20260811 02:57:46.775315 18157 ts_manager.cc:194] Registered new tserver with Master: 8825a306f0c64765891999d3baed45a2 (127.8.203.66:34901)
W20260811 02:57:46.884351 18787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:46.884606 18787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:46.884647 18787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:46.888609 18787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:46.888657 18787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:46.888708 18787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:46.893327 18787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:32997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:46.894250 18787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:46.895516 18787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:46.904790 18804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:46.904796 18796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:46.904906 18787 server_base.cc:1034] running on GCE node
W20260811 02:57:46.905071 18798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:46.905517 18787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:46.906117 18787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:46.907276 18787 hybrid_clock.cc:648] HybridClock initialized: now 1786417066907255 us; error 32 us; skew 500 ppm
I20260811 02:57:46.908955 18787 webserver.cc:460] Webserver started at http://127.8.203.67:39605/ using document root <none> and password file <none>
I20260811 02:57:46.909540 18787 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:46.909610 18787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:46.909799 18787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:46.911751 18787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "091e0a51f27643038622fb834181508e"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "a4cce446351d32075f7ebddc6828dc67"
server_key_iv: "df45c236a9cfbd64422ef12680e2775c"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.912294 18787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "091e0a51f27643038622fb834181508e"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "a4cce446351d32075f7ebddc6828dc67"
server_key_iv: "df45c236a9cfbd64422ef12680e2775c"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.915812 18787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 02:57:46.918043 18818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.919173 18787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:46.919271 18787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "091e0a51f27643038622fb834181508e"
format_stamp: "Formatted at 2026-08-11 02:57:46 on dist-test-slave-gjw7"
server_key: "a4cce446351d32075f7ebddc6828dc67"
server_key_iv: "df45c236a9cfbd64422ef12680e2775c"
server_key_version: "encryptionkey@0"
I20260811 02:57:46.919353 18787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:46.923599 18787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:46.924122 18787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:46.924248 18787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:46.924701 18787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:46.925539 18787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:46.925585 18787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.925621 18787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:46.925639 18787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:46.933804 18787 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43715
I20260811 02:57:46.933827 19045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43715 every 8 connection(s)
I20260811 02:57:46.934613 18787 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:46.940305 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.936290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:50317 (local address 127.8.203.126:32997)
0811 02:57:46.936425 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.936428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.937419 (+ 991us) server_negotiation.cc:407] Connection header received
0811 02:57:46.938677 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.938680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.938725 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.938803 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.939866 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.939870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.939873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.939885 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.939905 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.939912 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.939989 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.940123 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.940169 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:57:46.940752 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18787
I20260811 02:57:46.940857 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:46.941129 9005 external_mini_cluster.cc:1428] Setting key 8ee6ce6c1f37182d755497f64202f64d
I20260811 02:57:46.941334 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.936646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:32997 (local address 127.8.203.67:50317)
0811 02:57:46.937251 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.937318 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.938473 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.938989 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.938996 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.939081 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:46.939608 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:46.939610 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:46.939649 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:46.939715 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:46.939986 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:46.939994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:46.940001 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.940152 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.940382 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":96,"threads_started":1}
I20260811 02:57:46.942397 19046 heartbeater.cc:347] Connected to a master server at 127.8.203.126:32997
I20260811 02:57:46.942619 19046 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:46.943046 19046 heartbeater.cc:511] Master 127.8.203.126:32997 requested a full tablet report, sending...
I20260811 02:57:46.944015 18154 ts_manager.cc:194] Registered new tserver with Master: 091e0a51f27643038622fb834181508e (127.8.203.67:43715)
I20260811 02:57:46.954726 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:57:46.958098 18233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.956508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45728 (local address 127.8.203.126:32997)
0811 02:57:46.956615 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.956619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.956818 (+ 199us) server_negotiation.cc:407] Connection header received
0811 02:57:46.956949 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.956952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.956985 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.957047 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.957738 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.957742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.957744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.957757 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.957775 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.957783 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.957852 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.957940 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.957980 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 02:57:46.960781 18154 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:46.963752 18154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:46.974839 19075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.971691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43715 (local address 127.0.0.1:55384)
0811 02:57:46.972512 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.972561 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.972671 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.973386 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.973393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.973472 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:46.974039 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:46.974041 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:46.974062 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:46.974134 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:46.974508 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:46.974518 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:46.974524 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.974627 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.974698 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"spinlock_wait_cycles":5760,"thread_start_us":88,"threads_started":1}
I20260811 02:57:46.974884 19077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.971820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.8.203.67:43715)
0811 02:57:46.972920 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.972931 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.972956 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:57:46.973009 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.973014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.973208 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.973373 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.974311 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.974317 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.974326 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.974337 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.974375 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.974410 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.974512 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.974658 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.974728 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":99,"threads_started":1}
I20260811 02:57:46.975304 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.971810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:36653 (local address 127.0.0.1:38398)
0811 02:57:46.972524 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.972561 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.972671 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.974064 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.974067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.974081 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:46.974615 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:46.974617 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:46.974629 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:46.974653 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:46.975077 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:46.975080 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:46.975084 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.975147 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.975181 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"spinlock_wait_cycles":6528,"thread_start_us":112,"threads_started":1}
I20260811 02:57:46.975749 19080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.972420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:34901 (local address 127.0.0.1:47536)
0811 02:57:46.973176 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:46.973187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:46.973252 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:46.974064 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:46.974067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:46.974081 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:46.975017 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:46.975019 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:46.975029 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:46.975055 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:46.975536 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:46.975539 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:46.975542 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:57:46.975607 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:57:46.975639 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":93,"threads_started":1}
I20260811 02:57:46.975783 19078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.972160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.8.203.65:36653)
0811 02:57:46.972919 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.972932 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.972956 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:57:46.973009 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.973014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.973208 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.973381 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.974784 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.974791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.974803 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.974814 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.974853 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.974877 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.974972 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.975546 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.975623 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":117,"threads_started":1}
I20260811 02:57:46.976202 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:46.972824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.8.203.66:34901)
0811 02:57:46.973496 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:46.973506 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:46.973530 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:57:46.973580 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:46.973584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:46.973763 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:46.973904 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:46.975187 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:46.975194 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:46.975205 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:46.975216 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:46.975256 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:46.975284 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:46.975489 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:46.975978 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 02:57:46.976048 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":112,"threads_started":1}
I20260811 02:57:46.978139 18908 tablet_service.cc:1467] Processing CreateTablet for tablet d0c0e127987d46f3a78eff237bcebdc9 (DEFAULT_TABLE table=test-table [id=c28168d0dd204cf3b129291026e92426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:46.978593 18404 tablet_service.cc:1467] Processing CreateTablet for tablet d0c0e127987d46f3a78eff237bcebdc9 (DEFAULT_TABLE table=test-table [id=c28168d0dd204cf3b129291026e92426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:46.978593 18655 tablet_service.cc:1467] Processing CreateTablet for tablet d0c0e127987d46f3a78eff237bcebdc9 (DEFAULT_TABLE table=test-table [id=c28168d0dd204cf3b129291026e92426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:46.979249 18908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c0e127987d46f3a78eff237bcebdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:46.979598 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c0e127987d46f3a78eff237bcebdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:46.979598 18404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0c0e127987d46f3a78eff237bcebdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:46.985635 19082 tablet_bootstrap.cc:492] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Bootstrap starting.
I20260811 02:57:46.987249 19084 tablet_bootstrap.cc:492] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Bootstrap starting.
I20260811 02:57:46.987249 19083 tablet_bootstrap.cc:492] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: Bootstrap starting.
I20260811 02:57:46.988161 19082 tablet_bootstrap.cc:654] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:46.988862 19082 log.cc:824] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:46.989349 19084 tablet_bootstrap.cc:654] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:46.989349 19083 tablet_bootstrap.cc:654] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:46.990170 19083 log.cc:824] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:46.990170 19084 log.cc:824] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:46.993316 19082 tablet_bootstrap.cc:492] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: No bootstrap required, opened a new log
I20260811 02:57:46.993505 19082 ts_tablet_manager.cc:1397] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 02:57:46.993549 19084 tablet_bootstrap.cc:492] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: No bootstrap required, opened a new log
I20260811 02:57:46.993549 19083 tablet_bootstrap.cc:492] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: No bootstrap required, opened a new log
I20260811 02:57:46.993722 19084 ts_tablet_manager.cc:1397] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260811 02:57:46.993722 19083 ts_tablet_manager.cc:1397] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260811 02:57:46.996703 19082 raft_consensus.cc:348] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:46.996929 19082 raft_consensus.cc:374] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:46.996994 19082 raft_consensus.cc:729] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091e0a51f27643038622fb834181508e, State: Initialized, Role: FOLLOWER
I20260811 02:57:46.997432 19082 consensus_queue.cc:260] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:46.998016 19046 heartbeater.cc:503] Master 127.8.203.126:32997 was elected leader, sending a full tablet report...
I20260811 02:57:46.998031 19084 raft_consensus.cc:348] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:46.998015 19083 raft_consensus.cc:348] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:46.998246 19084 raft_consensus.cc:374] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:46.998246 19083 raft_consensus.cc:374] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:46.998292 19084 raft_consensus.cc:729] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8825a306f0c64765891999d3baed45a2, State: Initialized, Role: FOLLOWER
I20260811 02:57:46.998301 19083 raft_consensus.cc:729] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e518574da0947d096666f8c66cde604, State: Initialized, Role: FOLLOWER
I20260811 02:57:46.998386 19082 ts_tablet_manager.cc:1428] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:57:46.998692 19084 consensus_queue.cc:260] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:46.998692 19083 consensus_queue.cc:260] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:46.999603 19083 ts_tablet_manager.cc:1428] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:46.999604 19084 ts_tablet_manager.cc:1428] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260811 02:57:46.999912 18528 heartbeater.cc:503] Master 127.8.203.126:32997 was elected leader, sending a full tablet report...
I20260811 02:57:47.000761 18781 heartbeater.cc:503] Master 127.8.203.126:32997 was elected leader, sending a full tablet report...
W20260811 02:57:47.017086 18783 tablet.cc:2369] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:47.098347 18531 tablet.cc:2369] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:47.185505 19047 tablet.cc:2369] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:47.241430 19091 raft_consensus.cc:482] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:47.241652 19091 raft_consensus.cc:504] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:47.242810 19091 leader_election.cc:288] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e518574da0947d096666f8c66cde604 (127.8.203.65:36653), 8825a306f0c64765891999d3baed45a2 (127.8.203.66:34901)
I20260811 02:57:47.244724 19078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.243206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:38873 (local address 127.8.203.65:36653)
0811 02:57:47.243325 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:47.243329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:47.243344 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:57:47.243466 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:47.243469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:47.243504 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:47.243573 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:47.244301 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:47.244306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:47.244308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:47.244321 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:47.244341 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:47.244351 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:47.244439 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:47.244544 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 02:57:47.244586 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:57:47.244705 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.243064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:36653 (local address 127.8.203.67:38873)
0811 02:57:47.243219 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:47.243236 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:47.243333 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:47.243616 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:47.243619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:47.243635 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:47.244113 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:47.244115 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:47.244128 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:47.244157 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:47.244445 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:47.244449 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:47.244452 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:57:47.244539 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 02:57:47.244577 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 02:57:47.244808 19081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.243189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:60565 (local address 127.8.203.66:34901)
0811 02:57:47.243310 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:47.243315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:47.243686 (+ 371us) server_negotiation.cc:407] Connection header received
0811 02:57:47.243769 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:47.243772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:47.243808 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:47.243881 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:47.244450 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:47.244453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:47.244456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:47.244468 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:47.244492 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:47.244503 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:47.244583 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:47.244661 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 02:57:47.244696 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:57:47.244867 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.243064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:34901 (local address 127.8.203.67:60565)
0811 02:57:47.243586 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:47.243597 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:47.243689 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:47.243908 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:47.243911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:47.243924 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:47.244280 (+ 356us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:57:47.244282 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:47.244292 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:57:47.244318 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:47.244582 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:47.244585 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:47.244588 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:57:47.244664 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 02:57:47.244695 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":171,"thread_start_us":132,"threads_started":1}
I20260811 02:57:47.245424 18455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c0e127987d46f3a78eff237bcebdc9" candidate_uuid: "091e0a51f27643038622fb834181508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e518574da0947d096666f8c66cde604" is_pre_election: true
I20260811 02:57:47.245467 18690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c0e127987d46f3a78eff237bcebdc9" candidate_uuid: "091e0a51f27643038622fb834181508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8825a306f0c64765891999d3baed45a2" is_pre_election: true
I20260811 02:57:47.245692 18455 raft_consensus.cc:2454] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 091e0a51f27643038622fb834181508e in term 0.
I20260811 02:57:47.245718 18690 raft_consensus.cc:2454] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 091e0a51f27643038622fb834181508e in term 0.
I20260811 02:57:47.246165 18823 leader_election.cc:302] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091e0a51f27643038622fb834181508e, 1e518574da0947d096666f8c66cde604; no voters:
I20260811 02:57:47.246397 19091 raft_consensus.cc:2781] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:47.246481 19091 raft_consensus.cc:482] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:47.246529 19091 raft_consensus.cc:3037] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:47.247543 19091 raft_consensus.cc:504] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:47.247836 19091 leader_election.cc:288] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [CANDIDATE]: Term 1 election: Requested vote from peers 1e518574da0947d096666f8c66cde604 (127.8.203.65:36653), 8825a306f0c64765891999d3baed45a2 (127.8.203.66:34901)
I20260811 02:57:47.248168 18455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c0e127987d46f3a78eff237bcebdc9" candidate_uuid: "091e0a51f27643038622fb834181508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e518574da0947d096666f8c66cde604"
I20260811 02:57:47.248188 18690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c0e127987d46f3a78eff237bcebdc9" candidate_uuid: "091e0a51f27643038622fb834181508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8825a306f0c64765891999d3baed45a2"
I20260811 02:57:47.248289 18455 raft_consensus.cc:3037] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:47.248296 18690 raft_consensus.cc:3037] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:47.249325 18690 raft_consensus.cc:2454] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 091e0a51f27643038622fb834181508e in term 1.
I20260811 02:57:47.249383 18455 raft_consensus.cc:2454] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 091e0a51f27643038622fb834181508e in term 1.
I20260811 02:57:47.249639 18819 leader_election.cc:302] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091e0a51f27643038622fb834181508e, 8825a306f0c64765891999d3baed45a2; no voters:
I20260811 02:57:47.249830 19091 raft_consensus.cc:2781] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:47.250006 19091 raft_consensus.cc:686] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 1 LEADER]: Becoming Leader. State: Replica: 091e0a51f27643038622fb834181508e, State: Running, Role: LEADER
I20260811 02:57:47.250293 19091 consensus_queue.cc:237] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } }
I20260811 02:57:47.253564 18157 catalog_manager.cc:5495] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e reported cstate change: term changed from 0 to 1, leader changed from <none> to 091e0a51f27643038622fb834181508e (127.8.203.67). New cstate: current_term: 1 leader_uuid: "091e0a51f27643038622fb834181508e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091e0a51f27643038622fb834181508e" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 } health_report { overall_health: UNKNOWN } } }
I20260811 02:57:47.262284 19077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.261051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55390 (local address 127.8.203.67:43715)
0811 02:57:47.261168 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:47.261171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:47.261183 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:57:47.261317 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:47.261320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:47.261389 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:47.261453 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:47.262037 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:47.262039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:47.262041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:47.262047 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:47.262060 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:47.262066 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:47.262120 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:47.262168 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 02:57:47.262197 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:57:47.267829 18690 raft_consensus.cc:1261] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 1 FOLLOWER]: Refusing update from remote peer 091e0a51f27643038622fb834181508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:47.267829 18455 raft_consensus.cc:1261] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 1 FOLLOWER]: Refusing update from remote peer 091e0a51f27643038622fb834181508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:47.268564 19091 consensus_queue.cc:1035] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8825a306f0c64765891999d3baed45a2" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 34901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:47.268765 19102 consensus_queue.cc:1035] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e518574da0947d096666f8c66cde604" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:47.275566 19108 mvcc.cc:205] Tried to move back new op lower bound from 7317164307521884160 to 7317164307459493888. Current Snapshot: MvccSnapshot[applied={T|T < 7317164307521884160}]
I20260811 02:57:47.276844 19111 mvcc.cc:205] Tried to move back new op lower bound from 7317164307521884160 to 7317164307459493888. Current Snapshot: MvccSnapshot[applied={T|T < 7317164307521884160}]
I20260811 02:57:47.283731 18157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:47.283885 18157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:47.286247 18157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe1859368c604bc89392b16c0f00e49d: Sending DeleteTablet for 3 replicas of tablet d0c0e127987d46f3a78eff237bcebdc9
I20260811 02:57:47.286936 18404 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c0e127987d46f3a78eff237bcebdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:47 UTC) from {username='slave'} at 127.0.0.1:38398
I20260811 02:57:47.286993 18655 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c0e127987d46f3a78eff237bcebdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:47 UTC) from {username='slave'} at 127.0.0.1:47536
I20260811 02:57:47.287011 18908 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c0e127987d46f3a78eff237bcebdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:47 UTC) from {username='slave'} at 127.0.0.1:55384
I20260811 02:57:47.287474 19125 tablet_replica.cc:330] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: stopping tablet replica
I20260811 02:57:47.287470 19122 tablet_replica.cc:330] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604: stopping tablet replica
I20260811 02:57:47.287752 19125 raft_consensus.cc:2229] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:47.287752 19122 raft_consensus.cc:2229] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:47.288028 19122 raft_consensus.cc:2258] T d0c0e127987d46f3a78eff237bcebdc9 P 1e518574da0947d096666f8c66cde604 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:47.288066 19125 raft_consensus.cc:2258] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:47.288179 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18282
I20260811 02:57:47.308794 19124 tablet_replica.cc:330] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: stopping tablet replica
I20260811 02:57:47.309118 19124 raft_consensus.cc:2229] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:47.309428 19124 raft_consensus.cc:2258] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:47.314229 19125 ts_tablet_manager.cc:1905] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:47.314229 19124 ts_tablet_manager.cc:1905] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:47.317166 19124 ts_tablet_manager.cc:1918] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:47.317279 19124 log.cc:1196] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/d0c0e127987d46f3a78eff237bcebdc9
I20260811 02:57:47.317672 19124 ts_tablet_manager.cc:1939] T d0c0e127987d46f3a78eff237bcebdc9 P 8825a306f0c64765891999d3baed45a2: Deleting consensus metadata
I20260811 02:57:47.318985 18129 catalog_manager.cc:4841] TS 8825a306f0c64765891999d3baed45a2 (127.8.203.66:34901): tablet d0c0e127987d46f3a78eff237bcebdc9 (table test-table [id=c28168d0dd204cf3b129291026e92426]) successfully deleted
I20260811 02:57:47.319065 19125 ts_tablet_manager.cc:1918] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:47.319242 19125 log.cc:1196] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/d0c0e127987d46f3a78eff237bcebdc9
I20260811 02:57:47.319530 19125 ts_tablet_manager.cc:1939] T d0c0e127987d46f3a78eff237bcebdc9 P 091e0a51f27643038622fb834181508e: Deleting consensus metadata
I20260811 02:57:47.320582 18129 catalog_manager.cc:4841] TS 091e0a51f27643038622fb834181508e (127.8.203.67:43715): tablet d0c0e127987d46f3a78eff237bcebdc9 (table test-table [id=c28168d0dd204cf3b129291026e92426]) successfully deleted
W20260811 02:57:47.322922 18135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:36653 (error 108)
I20260811 02:57:47.323135 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18543
I20260811 02:57:47.341976 19080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.341645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:36653 (local address 127.0.0.1:38400)
0811 02:57:47.341790 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:47.341877 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:36653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 02:57:47.342197 18135 catalog_manager.cc:4584] TS 1e518574da0947d096666f8c66cde604 (127.8.203.65:36653): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0c0e127987d46f3a78eff237bcebdc9: Network error: Client connection negotiation failed: client connection to 127.8.203.65:36653: connect: Connection refused (error 111)
I20260811 02:57:47.348531 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18787
I20260811 02:57:47.361464 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18104
2026-08-11T02:57:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1190 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 02:57:47.391268 9005 mini_oidc.cc:152] Starting JWKS server
I20260811 02:57:47.393225 9005 webserver.cc:460] Webserver started at https://127.0.0.1:39737/ using document root <none> and password file <none>
I20260811 02:57:47.393462 9005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:57:47.393759 9005 webserver.cc:460] Webserver started at http://127.0.0.1:39823/ using document root <none> and password file <none>
2026-08-11T02:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:47Z Disabled control of system clock
I20260811 02:57:47.403353 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:37775
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:37775
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39737/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/testchainca.cert with env {}
W20260811 02:57:47.514676 19135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:47.514922 19135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:47.514961 19135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:47.518904 19135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:47.518956 19135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:47.518975 19135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:47.518999 19135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:47.519016 19135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:57:47.519039 19135 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:57:47.519054 19135 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39737/jwks.json
I20260811 02:57:47.524226 19135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:37775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:37775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39737/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:47.525170 19135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:47.526160 19135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:47.534513 19147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:47.534595 19135 server_base.cc:1034] running on GCE node
W20260811 02:57:47.534538 19151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:47.534538 19148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:47.535244 19135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:47.536319 19135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:47.537482 19135 hybrid_clock.cc:648] HybridClock initialized: now 1786417067537462 us; error 30 us; skew 500 ppm
I20260811 02:57:47.539258 19135 webserver.cc:460] Webserver started at http://127.8.203.126:46433/ using document root <none> and password file <none>
I20260811 02:57:47.539801 19135 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:47.539867 19135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:47.540069 19135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:47.542085 19135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "c7f573709204453c9b9250ce541c41ba"
format_stamp: "Formatted at 2026-08-11 02:57:47 on dist-test-slave-gjw7"
server_key: "6bfde9b928f580952710f33dd32d1141"
server_key_iv: "e943e77a2d126ed2f7a4e449b5327459"
server_key_version: "encryptionkey@0"
I20260811 02:57:47.542639 19135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "c7f573709204453c9b9250ce541c41ba"
format_stamp: "Formatted at 2026-08-11 02:57:47 on dist-test-slave-gjw7"
server_key: "6bfde9b928f580952710f33dd32d1141"
server_key_iv: "e943e77a2d126ed2f7a4e449b5327459"
server_key_version: "encryptionkey@0"
I20260811 02:57:47.546092 19135 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:57:47.548365 19161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:47.549618 19135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:47.549728 19135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "c7f573709204453c9b9250ce541c41ba"
format_stamp: "Formatted at 2026-08-11 02:57:47 on dist-test-slave-gjw7"
server_key: "6bfde9b928f580952710f33dd32d1141"
server_key_iv: "e943e77a2d126ed2f7a4e449b5327459"
server_key_version: "encryptionkey@0"
I20260811 02:57:47.549819 19135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:47.561805 19135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:47.562414 19135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:47.562573 19135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:47.568804 19135 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:37775
I20260811 02:57:47.568830 19257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:37775 every 8 connection(s)
I20260811 02:57:47.569684 19135 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:47.572320 19260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:47.577594 19260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba: Bootstrap starting.
I20260811 02:57:47.579083 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19135
I20260811 02:57:47.579198 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:47.579475 9005 external_mini_cluster.cc:1428] Setting key 41d7c39302dfaabf0d3ad917f9073b6b
I20260811 02:57:47.580188 19260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:47.581024 19260 log.cc:824] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:47.582947 19260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba: No bootstrap required, opened a new log
I20260811 02:57:47.586817 19260 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f573709204453c9b9250ce541c41ba" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37775 } }
I20260811 02:57:47.587098 19260 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:47.587157 19260 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f573709204453c9b9250ce541c41ba, State: Initialized, Role: FOLLOWER
I20260811 02:57:47.587591 19260 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f573709204453c9b9250ce541c41ba" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37775 } }
I20260811 02:57:47.587715 19260 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:47.587769 19260 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:47.587863 19260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:47.588086 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.580489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.8.203.126:37775)
0811 02:57:47.581079 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:47.581093 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:47.581128 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:57:47.581795 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:47.581813 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:47.582130 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:47.582411 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:47.582738 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.583709 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.584903 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.585413 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.585528 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:47.586327 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:47.586333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:47.586355 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:47.586725 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:47.586780 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:47.586799 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:47.586873 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:47.587035 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 02:57:47.587303 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":103,"threads_started":1}
I20260811 02:57:47.588757 19260 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f573709204453c9b9250ce541c41ba" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37775 } }
I20260811 02:57:47.589092 19260 leader_election.cc:302] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7f573709204453c9b9250ce541c41ba; no voters:
I20260811 02:57:47.589416 19260 leader_election.cc:288] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:47.589514 19276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:47.590019 19276 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [term 1 LEADER]: Becoming Leader. State: Replica: c7f573709204453c9b9250ce541c41ba, State: Running, Role: LEADER
I20260811 02:57:47.590332 19276 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f573709204453c9b9250ce541c41ba" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37775 } }
I20260811 02:57:47.590612 19260 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:47.590974 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:57:47.591868 19278 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7f573709204453c9b9250ce541c41ba. Latest consensus state: current_term: 1 leader_uuid: "c7f573709204453c9b9250ce541c41ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f573709204453c9b9250ce541c41ba" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37775 } } }
I20260811 02:57:47.591998 19278 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:47.592846 19277 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7f573709204453c9b9250ce541c41ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f573709204453c9b9250ce541c41ba" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37775 } } }
I20260811 02:57:47.592950 19277 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba [sys.catalog]: This master's current role is: LEADER
W20260811 02:57:47.595309 19307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:47.595389 19307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:47.595480 19302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:47.596343 19302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:47.602645 19302 catalog_manager.cc:1347] Generated new cluster ID: d6161f04bc7a439fb628ac1ab7dd4068
I20260811 02:57:47.602705 19302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:47.620352 19302 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:47.621318 19302 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:47.638221 19302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7f573709204453c9b9250ce541c41ba: Generated new TSK 0
I20260811 02:57:47.638907 19302 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 02:57:47.669957 19326 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 02:57:47.670106 19326 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 02:57:47.670605 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.648311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.8.203.126:37775)
0811 02:57:47.648433 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:47.648437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:47.648658 (+ 221us) server_negotiation.cc:407] Connection header received
0811 02:57:47.648859 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:47.648862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:47.648919 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:47.649017 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:57:47.649454 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.650254 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.651463 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.651806 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.651871 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:47.652314 (+ 443us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:57:47.670245 (+ 17931us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 02:57:47.670496 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34386: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 02:57:47.670674 19270 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34386: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 02:57:47.672191 19324 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:47.648231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:37775 (local address 127.0.0.1:34386)
0811 02:57:47.648567 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:47.648583 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:47.648715 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:47.649137 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:47.649141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:47.649155 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:57:47.649272 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:47.649278 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.650407 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.650411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:47.651322 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:47.651332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.651988 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.651991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:47.652081 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:47.652128 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:57:47.670464 (+ 18336us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 02:57:47.670705 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.203.126:37775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":98,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.203.126:37775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 02:57:47.672989 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19135
2026-08-11T02:57:47Z chronyd exiting
I20260811 02:57:47.718813 9005 test_util.cc:175] -----------------------------------------------
I20260811 02:57:47.718891 9005 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786417039021389-9005-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (329 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 02:57:47.719945 9005 mini_oidc.cc:152] Starting JWKS server
I20260811 02:57:47.720747 9005 webserver.cc:460] Webserver started at https://127.0.0.1:34761/ using document root <none> and password file <none>
I20260811 02:57:47.720825 9005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:57:47.721047 9005 webserver.cc:460] Webserver started at http://127.0.0.1:39631/ using document root <none> and password file <none>
2026-08-11T02:57:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:47Z Disabled control of system clock
I20260811 02:57:47.730278 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43191
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:42543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43191
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34761/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/testchainca.cert with env {}
W20260811 02:57:47.838303 19337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:47.838559 19337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:47.838613 19337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:47.842522 19337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:47.842573 19337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:47.842595 19337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:47.842615 19337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:47.842636 19337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:57:47.842665 19337 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:57:47.842685 19337 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34761/jwks.json
I20260811 02:57:47.847787 19337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34761/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:47.848755 19337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:47.849752 19337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:47.857740 19350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:47.857759 19352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:47.857841 19337 server_base.cc:1034] running on GCE node
W20260811 02:57:47.857757 19346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:47.858493 19337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:47.859469 19337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:47.860618 19337 hybrid_clock.cc:648] HybridClock initialized: now 1786417067860599 us; error 30 us; skew 500 ppm
I20260811 02:57:47.862331 19337 webserver.cc:460] Webserver started at http://127.8.203.126:34089/ using document root <none> and password file <none>
I20260811 02:57:47.862835 19337 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:47.862890 19337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:47.863052 19337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:47.864951 19337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "327d167dc7b540f287bc968cbff9f251"
format_stamp: "Formatted at 2026-08-11 02:57:47 on dist-test-slave-gjw7"
server_key: "9a8e0356bd099fab984351117b042722"
server_key_iv: "8596f796d811552d0fbb11d3e867b090"
server_key_version: "encryptionkey@0"
I20260811 02:57:47.865494 19337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "327d167dc7b540f287bc968cbff9f251"
format_stamp: "Formatted at 2026-08-11 02:57:47 on dist-test-slave-gjw7"
server_key: "9a8e0356bd099fab984351117b042722"
server_key_iv: "8596f796d811552d0fbb11d3e867b090"
server_key_version: "encryptionkey@0"
I20260811 02:57:47.868782 19337 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:57:47.871028 19359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:47.872148 19337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:47.872264 19337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "327d167dc7b540f287bc968cbff9f251"
format_stamp: "Formatted at 2026-08-11 02:57:47 on dist-test-slave-gjw7"
server_key: "9a8e0356bd099fab984351117b042722"
server_key_iv: "8596f796d811552d0fbb11d3e867b090"
server_key_version: "encryptionkey@0"
I20260811 02:57:47.872373 19337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:47.908439 19337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:47.909207 19337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:47.909431 19337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:47.916148 19337 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:43191
I20260811 02:57:47.916170 19476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:43191 every 8 connection(s)
I20260811 02:57:47.917068 19337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:47.920082 19480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:47.925457 19480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251: Bootstrap starting.
I20260811 02:57:47.926537 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19337
I20260811 02:57:47.926646 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:47.926942 9005 external_mini_cluster.cc:1428] Setting key b0a4297c9723b581b2697b3b512e0d08
I20260811 02:57:47.928114 19480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:47.928993 19480 log.cc:824] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:47.930945 19480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251: No bootstrap required, opened a new log
I20260811 02:57:47.934239 19480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "327d167dc7b540f287bc968cbff9f251" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43191 } }
I20260811 02:57:47.934468 19480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:47.934525 19480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 327d167dc7b540f287bc968cbff9f251, State: Initialized, Role: FOLLOWER
I20260811 02:57:47.934990 19480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "327d167dc7b540f287bc968cbff9f251" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43191 } }
I20260811 02:57:47.935138 19480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:47.935205 19480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:47.935297 19480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:47.935976 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.928020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39972 (local address 127.8.203.126:43191)
0811 02:57:47.928768 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:47.928787 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:47.928834 (+ 47us) server_negotiation.cc:407] Connection header received
0811 02:57:47.929692 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:47.929719 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:47.930099 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:47.930413 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:47.930823 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.931780 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.932654 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.933123 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.933269 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:47.934094 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:47.934099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:47.934112 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:47.934488 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:47.934558 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:47.934586 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:47.934677 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:47.934866 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 02:57:47.935145 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":135,"threads_started":1}
I20260811 02:57:47.936131 19480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "327d167dc7b540f287bc968cbff9f251" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43191 } }
I20260811 02:57:47.936486 19480 leader_election.cc:302] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 327d167dc7b540f287bc968cbff9f251; no voters:
I20260811 02:57:47.936810 19480 leader_election.cc:288] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:47.936898 19490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:47.937125 19490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [term 1 LEADER]: Becoming Leader. State: Replica: 327d167dc7b540f287bc968cbff9f251, State: Running, Role: LEADER
I20260811 02:57:47.937454 19490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "327d167dc7b540f287bc968cbff9f251" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43191 } }
I20260811 02:57:47.937944 19480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:47.939128 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:57:47.939134 19494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 327d167dc7b540f287bc968cbff9f251. Latest consensus state: current_term: 1 leader_uuid: "327d167dc7b540f287bc968cbff9f251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "327d167dc7b540f287bc968cbff9f251" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43191 } } }
I20260811 02:57:47.939361 19494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:47.939344 19491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "327d167dc7b540f287bc968cbff9f251" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "327d167dc7b540f287bc968cbff9f251" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43191 } } }
I20260811 02:57:47.939458 19491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:47.939739 19512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:47.942531 19512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:47.947906 19512 catalog_manager.cc:1347] Generated new cluster ID: a4b0a668ae694d1abdd5cdc1fa5a34f9
I20260811 02:57:47.947957 19512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:47.976054 19512 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:47.977032 19512 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:47.980403 19512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 327d167dc7b540f287bc968cbff9f251: Generated new TSK 0
I20260811 02:57:47.981010 19512 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 02:57:48.003098 19540 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 02:57:48.003165 19540 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 02:57:48.003813 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:47.988465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39980 (local address 127.8.203.126:43191)
0811 02:57:47.988577 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:47.988582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:47.988708 (+ 126us) server_negotiation.cc:407] Connection header received
0811 02:57:47.988842 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:47.988846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:47.988909 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:47.989015 (+ 106us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:57:47.989204 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.990041 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.990861 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.991216 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.991312 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:47.991596 (+ 284us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:57:48.003446 (+ 11850us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 02:57:48.003677 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39980: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 02:57:48.003865 19539 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:47.988369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43191 (local address 127.0.0.1:39980)
0811 02:57:47.988623 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:47.988635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:47.988726 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:47.989011 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:47.989013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:47.989019 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:57:47.989109 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:47.989113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.990134 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.990137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:47.990757 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:47.990764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:47.991312 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:47.991314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:47.991383 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:47.991405 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:57:48.003603 (+ 12198us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 02:57:48.003729 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.203.126:43191: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
W20260811 02:57:48.003883 19485 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39980: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34761/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 02:57:48.004418 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19337
2026-08-11T02:57:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (334 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 02:57:48.054522 9005 mini_oidc.cc:152] Starting JWKS server
I20260811 02:57:48.055357 9005 webserver.cc:460] Webserver started at https://127.0.0.1:36721/ using document root <none> and password file <none>
I20260811 02:57:48.055444 9005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:57:48.055661 9005 webserver.cc:460] Webserver started at http://127.0.0.1:33797/ using document root <none> and password file <none>
2026-08-11T02:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:48Z Disabled control of system clock
I20260811 02:57:48.066354 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45765
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:36541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:45765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36721/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 02:57:48.174930 19550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:48.175190 19550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:48.175242 19550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:48.179225 19550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:48.179286 19550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:48.179306 19550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:48.179327 19550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:48.179345 19550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:57:48.179365 19550 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:57:48.179384 19550 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36721/jwks.json
I20260811 02:57:48.184041 19550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:36541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:45765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36721/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:48.185092 19550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:48.186538 19550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:48.194640 19562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:48.194669 19561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:48.194648 19565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:48.194924 19550 server_base.cc:1034] running on GCE node
I20260811 02:57:48.195456 19550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:48.196511 19550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:48.197675 19550 hybrid_clock.cc:648] HybridClock initialized: now 1786417068197655 us; error 32 us; skew 500 ppm
I20260811 02:57:48.199352 19550 webserver.cc:460] Webserver started at http://127.8.203.126:43011/ using document root <none> and password file <none>
I20260811 02:57:48.199892 19550 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:48.199958 19550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:48.200142 19550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:48.202127 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "38c28e721f5b42d69265ea98a5289b45"
format_stamp: "Formatted at 2026-08-11 02:57:48 on dist-test-slave-gjw7"
server_key: "5c193c1e6715c44c05ed24a992b7613d"
server_key_iv: "80b9dfd3a24195b9ae6cc560d6403061"
server_key_version: "encryptionkey@0"
I20260811 02:57:48.202661 19550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "38c28e721f5b42d69265ea98a5289b45"
format_stamp: "Formatted at 2026-08-11 02:57:48 on dist-test-slave-gjw7"
server_key: "5c193c1e6715c44c05ed24a992b7613d"
server_key_iv: "80b9dfd3a24195b9ae6cc560d6403061"
server_key_version: "encryptionkey@0"
I20260811 02:57:48.206110 19550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:48.208258 19573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:48.209404 19550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:48.209522 19550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "38c28e721f5b42d69265ea98a5289b45"
format_stamp: "Formatted at 2026-08-11 02:57:48 on dist-test-slave-gjw7"
server_key: "5c193c1e6715c44c05ed24a992b7613d"
server_key_iv: "80b9dfd3a24195b9ae6cc560d6403061"
server_key_version: "encryptionkey@0"
I20260811 02:57:48.209609 19550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:48.225099 19550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:48.225696 19550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:48.225852 19550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:48.232187 19550 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:45765
I20260811 02:57:48.232199 19679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:45765 every 8 connection(s)
I20260811 02:57:48.233052 19550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:48.236085 19681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:48.241227 19681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45: Bootstrap starting.
I20260811 02:57:48.241284 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19550
I20260811 02:57:48.241418 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:48.241671 9005 external_mini_cluster.cc:1428] Setting key 763316344d3fee662fc70e83b89d4b17
I20260811 02:57:48.243593 19681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:48.244285 19681 log.cc:824] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:48.246186 19681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45: No bootstrap required, opened a new log
I20260811 02:57:48.250176 19681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c28e721f5b42d69265ea98a5289b45" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45765 } }
I20260811 02:57:48.250397 19681 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:48.250456 19681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c28e721f5b42d69265ea98a5289b45, State: Initialized, Role: FOLLOWER
I20260811 02:57:48.250604 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:48.242902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.8.203.126:45765)
0811 02:57:48.243477 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:48.243491 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:48.243525 (+ 34us) server_negotiation.cc:407] Connection header received
0811 02:57:48.244147 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:48.244165 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:48.244567 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:48.244931 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:48.245273 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.246251 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.247428 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.247886 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.247996 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:48.248823 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:48.248828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:48.248841 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:48.249234 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:48.249330 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:48.249360 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:48.249435 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:48.249592 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 02:57:48.249853 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
I20260811 02:57:48.250909 19681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c28e721f5b42d69265ea98a5289b45" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45765 } }
I20260811 02:57:48.251057 19681 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:48.251119 19681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:48.251219 19681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:48.252235 19681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c28e721f5b42d69265ea98a5289b45" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45765 } }
I20260811 02:57:48.252584 19681 leader_election.cc:302] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c28e721f5b42d69265ea98a5289b45; no voters:
I20260811 02:57:48.252897 19681 leader_election.cc:288] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:48.252978 19689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:48.253149 19689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [term 1 LEADER]: Becoming Leader. State: Replica: 38c28e721f5b42d69265ea98a5289b45, State: Running, Role: LEADER
I20260811 02:57:48.253374 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:57:48.253473 19689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c28e721f5b42d69265ea98a5289b45" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45765 } }
I20260811 02:57:48.254194 19681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:48.254663 19690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c28e721f5b42d69265ea98a5289b45. Latest consensus state: current_term: 1 leader_uuid: "38c28e721f5b42d69265ea98a5289b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c28e721f5b42d69265ea98a5289b45" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45765 } } }
I20260811 02:57:48.254778 19690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:48.255290 19692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c28e721f5b42d69265ea98a5289b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c28e721f5b42d69265ea98a5289b45" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45765 } } }
I20260811 02:57:48.255384 19692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:48.255656 19699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:48.258682 19699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:48.264186 19699 catalog_manager.cc:1347] Generated new cluster ID: e3da13a95f9d4b38b3685133e1b670da
I20260811 02:57:48.264245 19699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:48.287143 19699 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:48.288327 19699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:48.292341 19699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c28e721f5b42d69265ea98a5289b45: Generated new TSK 0
I20260811 02:57:48.293048 19699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:48.304934 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:48.300199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.8.203.126:45765)
0811 02:57:48.300314 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:48.300319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:48.300431 (+ 112us) server_negotiation.cc:407] Connection header received
0811 02:57:48.300554 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:48.300557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:48.300610 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:48.300713 (+ 103us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:57:48.300898 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.301754 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.302570 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.302908 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.303004 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:48.303232 (+ 228us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:57:48.304593 (+ 1361us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 02:57:48.304810 (+ 217us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46326: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 02:57:48.304992 19686 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46326: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 02:57:48.306528 19730 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:48.300085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45765 (local address 127.0.0.1:46326)
0811 02:57:48.300342 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:48.300353 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:48.300453 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:48.300713 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:48.300715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:48.300721 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:57:48.300812 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:48.300817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.301846 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.301849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:48.302467 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:48.302475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.303008 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.303010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:48.303065 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:48.303087 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:57:48.304761 (+ 1674us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 02:57:48.306363 (+ 1602us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.203.126:45765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":1437,"thread_start_us":85,"threads_started":1}
I20260811 02:57:48.306823 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19550
E20260811 02:57:48.327124 19732 webserver.cc:533] Webserver: sslize failed:
E20260811 02:57:48.327211 19732 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T02:57:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (304 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T02:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:48Z Disabled control of system clock
I20260811 02:57:48.368228 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:39001
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:39001
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/testchainca.cert with env {}
W20260811 02:57:48.475741 19741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:48.476015 19741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:48.476068 19741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:48.479947 19741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:48.480002 19741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:48.480022 19741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:48.480043 19741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:48.480062 19741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:57:48.480084 19741 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:57:48.480108 19741 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 02:57:48.485210 19741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:39001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:39001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:48.486207 19741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:48.487154 19741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:48.495136 19749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:48.495240 19741 server_base.cc:1034] running on GCE node
W20260811 02:57:48.495131 19751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:48.495127 19755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:48.495846 19741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:48.496870 19741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:48.498026 19741 hybrid_clock.cc:648] HybridClock initialized: now 1786417068498015 us; error 26 us; skew 500 ppm
I20260811 02:57:48.499752 19741 webserver.cc:460] Webserver started at http://127.8.203.126:39595/ using document root <none> and password file <none>
I20260811 02:57:48.500286 19741 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:48.500360 19741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:48.500567 19741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:48.502553 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "3074824536af444e9d9f061afe5c720f"
format_stamp: "Formatted at 2026-08-11 02:57:48 on dist-test-slave-gjw7"
server_key: "e0701396dffeb863233b44ab6c7b7dac"
server_key_iv: "99321a7948a94422fc7622ac9829faba"
server_key_version: "encryptionkey@0"
I20260811 02:57:48.503077 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "3074824536af444e9d9f061afe5c720f"
format_stamp: "Formatted at 2026-08-11 02:57:48 on dist-test-slave-gjw7"
server_key: "e0701396dffeb863233b44ab6c7b7dac"
server_key_iv: "99321a7948a94422fc7622ac9829faba"
server_key_version: "encryptionkey@0"
I20260811 02:57:48.506495 19741 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:48.508744 19770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:48.509912 19741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:48.510020 19741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "3074824536af444e9d9f061afe5c720f"
format_stamp: "Formatted at 2026-08-11 02:57:48 on dist-test-slave-gjw7"
server_key: "e0701396dffeb863233b44ab6c7b7dac"
server_key_iv: "99321a7948a94422fc7622ac9829faba"
server_key_version: "encryptionkey@0"
I20260811 02:57:48.510114 19741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:48.523705 19741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:48.524290 19741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:48.524461 19741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:48.530872 19741 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:39001
I20260811 02:57:48.530886 19866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:39001 every 8 connection(s)
I20260811 02:57:48.531736 19741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:48.532874 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19741
I20260811 02:57:48.532975 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:48.533233 9005 external_mini_cluster.cc:1428] Setting key ca5a39bcf5d4924909116e8146515786
I20260811 02:57:48.534773 19867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:48.540200 19867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f: Bootstrap starting.
I20260811 02:57:48.541955 19873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:48.534481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.8.203.126:39001)
0811 02:57:48.535112 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:48.535127 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:48.535162 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:57:48.535781 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:48.535796 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:48.536201 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:48.536503 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:48.536917 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.537966 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.538830 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.539284 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.539430 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:48.540293 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:48.540301 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:48.540317 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:57:48.540684 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:48.540737 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:57:48.540758 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:48.540852 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:48.540995 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 02:57:48.541234 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":125,"threads_started":1}
I20260811 02:57:48.542407 19867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:48.543076 19867 log.cc:824] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:48.544564 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:57:48.544940 19867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f: No bootstrap required, opened a new log
I20260811 02:57:48.548329 19867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3074824536af444e9d9f061afe5c720f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39001 } }
I20260811 02:57:48.548545 19867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:48.548604 19867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3074824536af444e9d9f061afe5c720f, State: Initialized, Role: FOLLOWER
I20260811 02:57:48.548981 19867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3074824536af444e9d9f061afe5c720f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39001 } }
I20260811 02:57:48.549108 19867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:48.549170 19867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:48.549270 19867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:48.550097 19867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3074824536af444e9d9f061afe5c720f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39001 } }
I20260811 02:57:48.550411 19867 leader_election.cc:302] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3074824536af444e9d9f061afe5c720f; no voters:
I20260811 02:57:48.550699 19867 leader_election.cc:288] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:48.550769 19878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:48.550971 19878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [term 1 LEADER]: Becoming Leader. State: Replica: 3074824536af444e9d9f061afe5c720f, State: Running, Role: LEADER
I20260811 02:57:48.551268 19878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3074824536af444e9d9f061afe5c720f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39001 } }
I20260811 02:57:48.551787 19867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:48.552238 19881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3074824536af444e9d9f061afe5c720f. Latest consensus state: current_term: 1 leader_uuid: "3074824536af444e9d9f061afe5c720f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3074824536af444e9d9f061afe5c720f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39001 } } }
I20260811 02:57:48.552413 19881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:48.552685 19879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3074824536af444e9d9f061afe5c720f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3074824536af444e9d9f061afe5c720f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 39001 } } }
I20260811 02:57:48.552794 19879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:48.553107 19887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:48.555637 19887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:48.560480 19887 catalog_manager.cc:1347] Generated new cluster ID: 0f662119ce144bf6b5394300a312efdc
I20260811 02:57:48.560544 19887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:48.585629 19887 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:48.586766 19887 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:48.595700 19887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3074824536af444e9d9f061afe5c720f: Generated new TSK 0
I20260811 02:57:48.596467 19887 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:48.631848 19873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:48.606712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46618 (local address 127.8.203.126:39001)
0811 02:57:48.606816 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:48.606820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:48.606975 (+ 155us) server_negotiation.cc:407] Connection header received
0811 02:57:48.607088 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:48.607091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:48.607147 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:48.607247 (+ 100us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:57:48.607436 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.608218 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.609073 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.609453 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.609549 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:48.609882 (+ 333us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:57:48.631415 (+ 21533us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 02:57:48.631699 (+ 284us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46618: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 02:57:48.631923 19873 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46618: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 02:57:48.631956 19920 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:48.606618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:39001 (local address 127.0.0.1:46618)
0811 02:57:48.606878 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:48.606890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:48.606986 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:48.607251 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:48.607253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:48.607258 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:57:48.607346 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:48.607351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.608340 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.608343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:48.608961 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:48.608969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:48.609597 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:48.609601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:48.609692 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:48.609716 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:57:48.631634 (+ 21918us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 02:57:48.631788 (+ 154us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.203.126:39001: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":84,"threads_started":1}
I20260811 02:57:48.632586 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19741
2026-08-11T02:57:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (307 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (127 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (121 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:48 dist-test-slave-gjw7 krb5kdc[19945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:48 dist-test-slave-gjw7 krb5kdc[19945](info): set up 2 sockets
Aug 11 02:57:48 dist-test-slave-gjw7 krb5kdc[19945](info): commencing operation
krb5kdc: starting...
W20260811 02:57:50.089118 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:50 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:57:50.208127 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46191
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:50.317003 19975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:50.317239 19975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:50.317317 19975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:50.321177 19975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:50.321226 19975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:50.321251 19975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:50.321272 19975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:50.321310 19975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:50.326330 19975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:50.327255 19975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:50.328177 19975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:50.336090 19987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:50.336092 19983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:50.336270 19975 server_base.cc:1034] running on GCE node
W20260811 02:57:50.336091 19984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:50.336880 19975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:50.337924 19975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:50.339099 19975 hybrid_clock.cc:648] HybridClock initialized: now 1786417070339072 us; error 39 us; skew 500 ppm
Aug 11 02:57:50 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:50.344152 19975 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:57:50.344995 19975 webserver.cc:460] Webserver started at http://127.8.203.126:33005/ using document root <none> and password file <none>
I20260811 02:57:50.345551 19975 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:50.345618 19975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:50.345810 19975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:50.347731 19975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "0b1aa1939b184eca8dbb49ec22b0a03b"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "857904614320a9eb2a8ac911f9ebface"
server_key_iv: "3f338d9f56353f02908f76d69173b9e0"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.348273 19975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "0b1aa1939b184eca8dbb49ec22b0a03b"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "857904614320a9eb2a8ac911f9ebface"
server_key_iv: "3f338d9f56353f02908f76d69173b9e0"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.351753 19975 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 02:57:50.354063 19996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.355203 19975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:50.355314 19975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "0b1aa1939b184eca8dbb49ec22b0a03b"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "857904614320a9eb2a8ac911f9ebface"
server_key_iv: "3f338d9f56353f02908f76d69173b9e0"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.355404 19975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:50.371014 19975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:50.373581 19975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:50.373745 19975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:50.380261 19975 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:46191
I20260811 02:57:50.380270 20093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:46191 every 8 connection(s)
I20260811 02:57:50.381127 19975 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:50.383332 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19975
I20260811 02:57:50.383426 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:50.383677 9005 external_mini_cluster.cc:1428] Setting key af532e4b690a83c100a0e33bd3c1d0e4
I20260811 02:57:50.383782 20094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:50.391074 20094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b: Bootstrap starting.
Aug 11 02:57:50 dist-test-slave-gjw7 krb5kdc[19945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:50.393497 20094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:50.394412 20094 log.cc:824] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:50.396584 20094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b: No bootstrap required, opened a new log
I20260811 02:57:50.398536 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:50.385019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.8.203.126:46191)
0811 02:57:50.385448 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:50.385466 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:50.385508 (+ 42us) server_negotiation.cc:407] Connection header received
0811 02:57:50.386391 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:50.386415 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:50.386820 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:50.387202 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:50.387854 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.389332 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.390184 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.390878 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.391042 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:50.393403 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:50.393411 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:50.393428 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:50.393463 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:50.395239 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:50.395644 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:50.395650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:50.395655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:50.395816 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:50.396068 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:50.396071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:50.396073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:50.396566 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:50.396769 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:50.397037 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 02:57:50.397364 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":110,"threads_started":1}
I20260811 02:57:50.400389 20094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46191 } }
I20260811 02:57:50.400642 20094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:50.400701 20094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1aa1939b184eca8dbb49ec22b0a03b, State: Initialized, Role: FOLLOWER
I20260811 02:57:50.401095 20094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46191 } }
I20260811 02:57:50.401233 20094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:50.401284 20094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:50.401404 20094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:50.402269 20094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46191 } }
I20260811 02:57:50.402622 20094 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b1aa1939b184eca8dbb49ec22b0a03b; no voters:
I20260811 02:57:50.402953 20094 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:50.403033 20104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:50.403239 20104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [term 1 LEADER]: Becoming Leader. State: Replica: 0b1aa1939b184eca8dbb49ec22b0a03b, State: Running, Role: LEADER
I20260811 02:57:50.403563 20104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46191 } }
I20260811 02:57:50.404126 20094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:50.405000 20106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b1aa1939b184eca8dbb49ec22b0a03b. Latest consensus state: current_term: 1 leader_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46191 } } }
I20260811 02:57:50.405335 20106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:50.405210 20105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1aa1939b184eca8dbb49ec22b0a03b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46191 } } }
I20260811 02:57:50.405608 20105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:50.405709 20120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:50.408327 20120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:50.413829 20120 catalog_manager.cc:1347] Generated new cluster ID: f3264f20f67d4ec9928c0662017a2948
I20260811 02:57:50.413892 20120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:50.432551 20120 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:50.433753 20120 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:50.439646 20120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b1aa1939b184eca8dbb49ec22b0a03b: Generated new TSK 0
I20260811 02:57:50.440361 20120 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:57:50.471944 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46191
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:50.581795 20155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:50.582033 20155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:50.582072 20155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:50.585960 20155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:50.586005 20155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:50.586061 20155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:50.591066 20155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:50.591995 20155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:50.592934 20155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:50.602259 20170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:50.602314 20155 server_base.cc:1034] running on GCE node
W20260811 02:57:50.602267 20171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:50.602267 20173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:50.602823 20155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:50.603426 20155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:50.604597 20155 hybrid_clock.cc:648] HybridClock initialized: now 1786417070604570 us; error 37 us; skew 500 ppm
Aug 11 02:57:50 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:50.607295 20155 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:57:50.608091 20155 webserver.cc:460] Webserver started at http://127.8.203.65:38133/ using document root <none> and password file <none>
I20260811 02:57:50.608597 20155 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:50.608654 20155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:50.608814 20155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:50.610821 20155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "0d1e2b476c604b91a145d8b667eea1ab"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "2798ec32623826738bb0edd6101629b2"
server_key_iv: "063ae7568cf682f972ca9c8935ca7309"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.611323 20155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "0d1e2b476c604b91a145d8b667eea1ab"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "2798ec32623826738bb0edd6101629b2"
server_key_iv: "063ae7568cf682f972ca9c8935ca7309"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.614871 20155 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:57:50.617345 20186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.618517 20155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:50.618610 20155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "0d1e2b476c604b91a145d8b667eea1ab"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "2798ec32623826738bb0edd6101629b2"
server_key_iv: "063ae7568cf682f972ca9c8935ca7309"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.618691 20155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:50.639225 20155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:50.641939 20155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:50.642082 20155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:50.642614 20155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:50.643364 20155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:50.643410 20155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.643447 20155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:50.643467 20155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.651876 20155 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:45969
I20260811 02:57:50.651906 20405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:45969 every 8 connection(s)
I20260811 02:57:50.652720 20155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:50.658033 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20155
I20260811 02:57:50.658161 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:50.658452 9005 external_mini_cluster.cc:1428] Setting key 0db2c61848120c59a19ac7fc3a3c0398
Aug 11 02:57:50 dist-test-slave-gjw7 krb5kdc[19945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:50.665997 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:50.654431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:36581 (local address 127.8.203.126:46191)
0811 02:57:50.654571 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:50.654576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:50.655412 (+ 836us) server_negotiation.cc:407] Connection header received
0811 02:57:50.656725 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:50.656729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:50.656775 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:50.656847 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:50.657998 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.658755 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.660035 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.660273 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.660366 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:50.662705 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:50.662708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:50.662710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:50.662733 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:50.663999 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:50.664480 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:50.664482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:50.664484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:50.664581 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:50.664908 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:50.664912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:50.664913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:50.665089 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:50.665174 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:50.665716 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 02:57:50.665839 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:57:50.666777 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:50.654808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46191 (local address 127.8.203.65:36581)
0811 02:57:50.655256 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:50.655299 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:50.656538 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:50.657002 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:50.657008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:50.657388 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:50.657852 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:50.657866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.658892 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.658896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:50.659918 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:50.659927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.660428 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.660432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:50.660582 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:50.661090 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:50.661106 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:50.662542 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:50.664115 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:50.664120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:50.664135 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:50.664378 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:50.664663 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:50.664666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:50.664669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:50.664804 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:50.665199 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:50.665205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:50.665485 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 02:57:50.665649 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 02:57:50.665959 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":111,"threads_started":1}
I20260811 02:57:50.667922 20407 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46191
I20260811 02:57:50.668166 20407 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:57:50.668700 20407 heartbeater.cc:511] Master 127.8.203.126:46191 requested a full tablet report, sending...
I20260811 02:57:50.670171 20037 ts_manager.cc:194] Registered new tserver with Master: 0d1e2b476c604b91a145d8b667eea1ab (127.8.203.65:45969)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 02:57:50.671375 20037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:36581
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:57:50.685251 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46191
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:50.795112 20420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:50.795353 20420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:50.795399 20420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:50.799316 20420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:50.799360 20420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:50.799419 20420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:50.804443 20420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:50.805464 20420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:50.806463 20420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:50.815838 20431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:50.815846 20437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:50.815836 20432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:50.816170 20420 server_base.cc:1034] running on GCE node
I20260811 02:57:50.816561 20420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:50.817214 20420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:50.818384 20420 hybrid_clock.cc:648] HybridClock initialized: now 1786417070818350 us; error 37 us; skew 500 ppm
Aug 11 02:57:50 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:50.820979 20420 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:57:50.821880 20420 webserver.cc:460] Webserver started at http://127.8.203.66:37385/ using document root <none> and password file <none>
I20260811 02:57:50.822436 20420 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:50.822499 20420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:50.822695 20420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:50.824661 20420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "cad506fcde974c00975d0ba4b88fb6be"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "634b460a4291d840930be989f7f9fbf1"
server_key_iv: "cf9a2fd6dc72d056762525bca1f8430b"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.825199 20420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "cad506fcde974c00975d0ba4b88fb6be"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "634b460a4291d840930be989f7f9fbf1"
server_key_iv: "cf9a2fd6dc72d056762525bca1f8430b"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.828683 20420 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:57:50.831032 20449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.832194 20420 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 02:57:50.832291 20420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "cad506fcde974c00975d0ba4b88fb6be"
format_stamp: "Formatted at 2026-08-11 02:57:50 on dist-test-slave-gjw7"
server_key: "634b460a4291d840930be989f7f9fbf1"
server_key_iv: "cf9a2fd6dc72d056762525bca1f8430b"
server_key_version: "encryptionkey@0"
I20260811 02:57:50.832393 20420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:50.847982 20420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:50.850618 20420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:50.850785 20420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:50.851347 20420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:50.852134 20420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:50.852178 20420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.852221 20420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:50.852250 20420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:50.860620 20420 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:34961
I20260811 02:57:50.860651 20675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:34961 every 8 connection(s)
I20260811 02:57:50.861498 20420 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:57:50 dist-test-slave-gjw7 krb5kdc[19945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417070, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:50.871551 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20420
I20260811 02:57:50.871670 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:50.872000 9005 external_mini_cluster.cc:1428] Setting key 49616c2068bbf26ab921c3a3ddd3d1db
I20260811 02:57:50.874362 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:50.863225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:32905 (local address 127.8.203.126:46191)
0811 02:57:50.863365 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:50.863369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:50.864156 (+ 787us) server_negotiation.cc:407] Connection header received
0811 02:57:50.865531 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:50.865534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:50.865595 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:50.865673 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:50.866821 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.867343 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.868448 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.868671 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.868759 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:50.871081 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:50.871083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:50.871085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:50.871108 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:50.872425 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:50.872890 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:50.872893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:50.872894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:50.872972 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:50.873305 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:50.873309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:50.873311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:50.873503 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:50.873591 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:50.874084 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 02:57:50.874206 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:57:50.875173 20683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:50.863579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46191 (local address 127.8.203.66:32905)
0811 02:57:50.864007 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:50.864048 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:50.865281 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:50.865857 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:50.865864 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:50.866208 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:50.866671 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:50.866685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.867475 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.867478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:50.868322 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:50.868332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:50.868817 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:50.868821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:50.868980 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:50.869512 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:50.869533 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:50.870915 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:50.872536 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:50.872541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:50.872555 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:50.872789 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:50.873060 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:50.873063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:50.873065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:50.873195 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:50.873609 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:50.873615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:50.873847 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 02:57:50.874015 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 02:57:50.874317 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260811 02:57:50.876338 20678 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46191
I20260811 02:57:50.876560 20678 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:50.877007 20678 heartbeater.cc:511] Master 127.8.203.126:46191 requested a full tablet report, sending...
I20260811 02:57:50.878088 20037 ts_manager.cc:194] Registered new tserver with Master: cad506fcde974c00975d0ba4b88fb6be (127.8.203.66:34961)
I20260811 02:57:50.878851 20037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:32905
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:57:50.898028 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46191
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:51.009400 20690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:51.009632 20690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:51.009672 20690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:51.013625 20690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:51.013693 20690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:51.013751 20690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:51.018920 20690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:46191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:51.019897 20690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:51.020852 20690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:51.030457 20690 server_base.cc:1034] running on GCE node
W20260811 02:57:51.030594 20708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:51.030393 20703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:51.030373 20701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:51.031064 20690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:51.031629 20690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:51.032785 20690 hybrid_clock.cc:648] HybridClock initialized: now 1786417071032771 us; error 25 us; skew 500 ppm
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.035523 20690 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:57:51.036437 20690 webserver.cc:460] Webserver started at http://127.8.203.67:33473/ using document root <none> and password file <none>
I20260811 02:57:51.036942 20690 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:51.036998 20690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:51.037165 20690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:51.039209 20690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "324d98e576eb4116a04c2992295c3f31"
format_stamp: "Formatted at 2026-08-11 02:57:51 on dist-test-slave-gjw7"
server_key: "a40aa24b8d4d2ac31eeaffe506c90b19"
server_key_iv: "8d60b1512d3833ca8fee9e94b53b9152"
server_key_version: "encryptionkey@0"
I20260811 02:57:51.039716 20690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "324d98e576eb4116a04c2992295c3f31"
format_stamp: "Formatted at 2026-08-11 02:57:51 on dist-test-slave-gjw7"
server_key: "a40aa24b8d4d2ac31eeaffe506c90b19"
server_key_iv: "8d60b1512d3833ca8fee9e94b53b9152"
server_key_version: "encryptionkey@0"
I20260811 02:57:51.043267 20690 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:51.045567 20722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:51.046697 20690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:51.046806 20690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "324d98e576eb4116a04c2992295c3f31"
format_stamp: "Formatted at 2026-08-11 02:57:51 on dist-test-slave-gjw7"
server_key: "a40aa24b8d4d2ac31eeaffe506c90b19"
server_key_iv: "8d60b1512d3833ca8fee9e94b53b9152"
server_key_version: "encryptionkey@0"
I20260811 02:57:51.046896 20690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:51.059866 20690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:51.062824 20690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:51.062989 20690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:51.063535 20690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:51.064340 20690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:51.064397 20690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:51.064452 20690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:51.064481 20690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:51.074419 20690 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:44679
I20260811 02:57:51.074520 20931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:44679 every 8 connection(s)
I20260811 02:57:51.075489 20690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:57:51.084784 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20690
I20260811 02:57:51.084910 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:51.085218 9005 external_mini_cluster.cc:1428] Setting key 8e208861a76700e934c0d5cf2ce32133
I20260811 02:57:51.088716 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:51.077329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:50227 (local address 127.8.203.126:46191)
0811 02:57:51.077464 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:51.077469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:51.078315 (+ 846us) server_negotiation.cc:407] Connection header received
0811 02:57:51.079650 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:51.079653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:51.079701 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:51.079773 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:51.080931 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.081527 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.082625 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.082844 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.082919 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:51.085349 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:51.085353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:51.085356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:51.085383 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:51.086950 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.087494 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.087497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.087498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.087578 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.087858 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.087860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.087861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.088024 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:51.088093 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:51.088506 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 02:57:51.088599 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 02:57:51.089509 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:51.077676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46191 (local address 127.8.203.67:50227)
0811 02:57:51.078131 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:51.078201 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:51.079443 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:51.079931 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:51.079937 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:51.080310 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:51.080793 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:51.080807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.081648 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.081651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:51.082515 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:51.082525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.082963 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.082966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:51.083097 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:51.083625 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:51.083641 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:51.085159 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:51.087062 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:51.087066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:51.087078 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:51.087394 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:51.087656 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:51.087659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:51.087660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:51.087788 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:51.088098 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:51.088102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:51.088312 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 02:57:51.088469 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 02:57:51.088701 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260811 02:57:51.090495 20935 heartbeater.cc:347] Connected to a master server at 127.8.203.126:46191
I20260811 02:57:51.090715 20935 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:51.091157 20935 heartbeater.cc:511] Master 127.8.203.126:46191 requested a full tablet report, sending...
I20260811 02:57:51.092105 20037 ts_manager.cc:194] Registered new tserver with Master: 324d98e576eb4116a04c2992295c3f31 (127.8.203.67:44679)
I20260811 02:57:51.092628 20037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:50227
I20260811 02:57:51.098980 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/test-admin.keytab.
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.115029 9005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 02:57:51.117898 20951 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:51.116368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40712 (local address 127.8.203.65:45969)
0811 02:57:51.116762 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:51.116775 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:51.116805 (+ 30us) server_negotiation.cc:407] Connection header received
0811 02:57:51.116861 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:51.116867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:51.117066 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:51.117183 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:51.117705 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40712: BlockingRecv error: recv got EOF from 127.0.0.1:40712 (error 108)
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":142,"threads_started":1}
W20260811 02:57:51.117995 9005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5cc' of type FILE
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.118949 9005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.119772 9005 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:51.128669 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:51.120563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.8.203.65:45969)
0811 02:57:51.120680 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:51.120684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:51.120745 (+ 61us) server_negotiation.cc:407] Connection header received
0811 02:57:51.120862 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:51.120865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:51.120915 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:51.120977 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:51.121539 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.122373 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.123235 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.123601 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.123681 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:51.125693 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:51.125703 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:51.125716 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:51.125740 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:51.127247 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.127610 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.127614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.127616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.127688 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.127875 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.127877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.127878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.128091 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:51.128238 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:51.128375 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 02:57:51.128518 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 02:57:51.132683 20951 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:51.131645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.8.203.65:45969)
0811 02:57:51.131759 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:51.131762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:51.131909 (+ 147us) server_negotiation.cc:407] Connection header received
0811 02:57:51.132021 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:51.132023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:51.132056 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:51.132113 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:51.132591 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40726: BlockingRecv error: recv got EOF from 127.0.0.1:40726 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 02:57:51.132788 9005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5cc' of type FILE
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.133682 9005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.134462 9005 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:51.141894 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:51.135204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.8.203.65:45969)
0811 02:57:51.135298 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:51.135302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:51.135417 (+ 115us) server_negotiation.cc:407] Connection header received
0811 02:57:51.135488 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:51.135490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:51.135521 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:51.135579 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:51.135982 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.136526 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.137231 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.137495 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.137570 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:51.139405 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:51.139408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:51.139410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:51.139431 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:51.140651 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.140963 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.140965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.140966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.141031 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.141213 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.141215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.141216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.141396 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:51.141486 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:51.141658 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 02:57:51.141751 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 02:57:51.144306 20951 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:57:51.143456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.8.203.65:45969)
0811 02:57:51.143571 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:51.143575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:51.143645 (+ 70us) server_negotiation.cc:407] Connection header received
0811 02:57:51.143748 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:51.143751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:51.143789 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:51.143852 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:51.144218 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40754: BlockingRecv error: recv got EOF from 127.0.0.1:40754 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 02:57:51.144426 9005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786417039021389-9005-0/krb5kdc/krb5cc' of type FILE
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.145331 9005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:51.146147 9005 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[19945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 02:57:51.153935 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:51.146841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.8.203.65:45969)
0811 02:57:51.146949 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:51.146953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:51.147011 (+ 58us) server_negotiation.cc:407] Connection header received
0811 02:57:51.147115 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:51.147118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:51.147154 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:51.147219 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:51.147687 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.148154 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.148909 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:51.149121 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:51.149205 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:51.151172 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:51.151176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:51.151179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:51.151200 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:51.152423 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.152811 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.152814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.152816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.152897 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:51.153151 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:51.153154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:51.153156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:51.153348 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:51.153460 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:51.153701 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 02:57:51.153801 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:57:51.154884 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20155
I20260811 02:57:51.175467 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20420
I20260811 02:57:51.201639 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20690
I20260811 02:57:51.219449 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19975
2026-08-11T02:57:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2334 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[20971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[20971](info): set up 2 sockets
Aug 11 02:57:51 dist-test-slave-gjw7 krb5kdc[20971](info): commencing operation
krb5kdc: starting...
W20260811 02:57:52.420603 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:57:52 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:57:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:57:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab.
Entry for principal oryx/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab.
I20260811 02:57:52.536008 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43163
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab
--principal=oryx/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:52.643754 20997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:52.644004 20997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:52.644055 20997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:52.647985 20997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:52.648036 20997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:52.648062 20997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:52.648083 20997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:52.648102 20997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:52.653077 20997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab
--principal=oryx/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:52.654032 20997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:52.654970 20997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:52.662969 20997 server_base.cc:1034] running on GCE node
W20260811 02:57:52.662920 21007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:52.662953 21011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:52.662920 21006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:52.663658 20997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:52.664615 20997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:52.665763 20997 hybrid_clock.cc:648] HybridClock initialized: now 1786417072665751 us; error 31 us; skew 500 ppm
Aug 11 02:57:52 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:52.668254 20997 init.cc:377] Logged in from keytab as oryx/127.8.203.126@KRBTEST.COM (short username oryx)
I20260811 02:57:52.669066 20997 webserver.cc:460] Webserver started at http://127.8.203.126:40771/ using document root <none> and password file <none>
I20260811 02:57:52.669613 20997 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:52.669679 20997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:52.669862 20997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:52.671742 20997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e"
format_stamp: "Formatted at 2026-08-11 02:57:52 on dist-test-slave-gjw7"
server_key: "82583e3da8ca51380ddec2263d54e91a"
server_key_iv: "e73d5b018f2d1d0871724797aada1663"
server_key_version: "encryptionkey@0"
I20260811 02:57:52.672266 20997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e"
format_stamp: "Formatted at 2026-08-11 02:57:52 on dist-test-slave-gjw7"
server_key: "82583e3da8ca51380ddec2263d54e91a"
server_key_iv: "e73d5b018f2d1d0871724797aada1663"
server_key_version: "encryptionkey@0"
I20260811 02:57:52.675753 20997 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:52.678004 21025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:52.679098 20997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:52.679207 20997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e"
format_stamp: "Formatted at 2026-08-11 02:57:52 on dist-test-slave-gjw7"
server_key: "82583e3da8ca51380ddec2263d54e91a"
server_key_iv: "e73d5b018f2d1d0871724797aada1663"
server_key_version: "encryptionkey@0"
I20260811 02:57:52.679297 20997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:52.703948 20997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:52.706591 20997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:52.706766 20997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:52.713454 20997 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:43163
I20260811 02:57:52.713733 21126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:43163 every 8 connection(s)
I20260811 02:57:52.714403 20997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:57:52.717685 21127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:52.721122 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20997
I20260811 02:57:52.721256 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:57:52.721591 9005 external_mini_cluster.cc:1428] Setting key a872141782e07b1227f4e80c177ec330
I20260811 02:57:52.723599 21127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Bootstrap starting.
I20260811 02:57:52.726007 21127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:52.726742 21127 log.cc:824] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:52.728752 21127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: No bootstrap required, opened a new log
Aug 11 02:57:52 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:57:52.732012 21127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } }
I20260811 02:57:52.732225 21127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:52.732285 21127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2eba647abc4dcbb6aeb7ae4ee2587e, State: Initialized, Role: FOLLOWER
I20260811 02:57:52.732708 21127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } }
I20260811 02:57:52.732851 21127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:57:52.732918 21127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:57:52.733016 21127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:52.734014 21127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } }
I20260811 02:57:52.734355 21127 leader_election.cc:302] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f2eba647abc4dcbb6aeb7ae4ee2587e; no voters:
I20260811 02:57:52.734694 21127 leader_election.cc:288] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:57:52.734799 21137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:52.735037 21137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 1 LEADER]: Becoming Leader. State: Replica: 1f2eba647abc4dcbb6aeb7ae4ee2587e, State: Running, Role: LEADER
I20260811 02:57:52.735399 21137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } }
I20260811 02:57:52.735998 21127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:52.736110 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:52.722920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43842 (local address 127.8.203.126:43163)
0811 02:57:52.723296 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:52.723311 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:52.723348 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:57:52.724062 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:52.724082 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:52.724420 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:52.724682 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:52.725554 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:52.726586 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:52.727574 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:52.728027 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:52.728136 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:52.730833 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:52.730839 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:52.730851 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:52.730886 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:52.732908 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:52.733404 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:52.733410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:52.733414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:52.733522 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:52.733801 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:52.733803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:52.733804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:52.734175 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:52.734329 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:52.734670 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 02:57:52.735126 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
I20260811 02:57:52.737788 21138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f2eba647abc4dcbb6aeb7ae4ee2587e. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } }
I20260811 02:57:52.737907 21138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:52.738132 21138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } }
I20260811 02:57:52.738348 21138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
W20260811 02:57:52.740518 21159 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:52.740581 21159 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:52.740666 21161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:52.741451 21161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:52.746526 21161 catalog_manager.cc:1347] Generated new cluster ID: 63496041b446402299cdb4f2f54f28be
I20260811 02:57:52.746588 21161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:52.766588 21161 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:57:52.768008 21161 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:52.776789 21161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Generated new TSK 0
I20260811 02:57:52.777462 21161 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab.
Entry for principal oryx/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab.
I20260811 02:57:52.808274 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43163
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab
--principal=oryx/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:52.923728 21170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:52.923988 21170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:52.924032 21170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:52.927929 21170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:52.927980 21170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:52.928043 21170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:52.932963 21170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab
--principal=oryx/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:52.933931 21170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:52.934904 21170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:52.944072 21180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:52.944063 21177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:52.944252 21170 server_base.cc:1034] running on GCE node
W20260811 02:57:52.944051 21182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:52.944628 21170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:52.945202 21170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:52.946368 21170 hybrid_clock.cc:648] HybridClock initialized: now 1786417072946336 us; error 35 us; skew 500 ppm
Aug 11 02:57:52 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:52.948863 21170 init.cc:377] Logged in from keytab as oryx/127.8.203.65@KRBTEST.COM (short username oryx)
I20260811 02:57:52.949710 21170 webserver.cc:460] Webserver started at http://127.8.203.65:40291/ using document root <none> and password file <none>
I20260811 02:57:52.950227 21170 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:52.950292 21170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:52.950481 21170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:52.952378 21170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "bfd1d5297db24ff58c7db4a7df308b7d"
format_stamp: "Formatted at 2026-08-11 02:57:52 on dist-test-slave-gjw7"
server_key: "3f78d306ddac21c2aafb9240f6249ea6"
server_key_iv: "f4ff57a36e6520cd1efeb1033141a9e9"
server_key_version: "encryptionkey@0"
I20260811 02:57:52.952908 21170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "bfd1d5297db24ff58c7db4a7df308b7d"
format_stamp: "Formatted at 2026-08-11 02:57:52 on dist-test-slave-gjw7"
server_key: "3f78d306ddac21c2aafb9240f6249ea6"
server_key_iv: "f4ff57a36e6520cd1efeb1033141a9e9"
server_key_version: "encryptionkey@0"
I20260811 02:57:52.956350 21170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:52.958676 21195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:52.959753 21170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:52.959853 21170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "bfd1d5297db24ff58c7db4a7df308b7d"
format_stamp: "Formatted at 2026-08-11 02:57:52 on dist-test-slave-gjw7"
server_key: "3f78d306ddac21c2aafb9240f6249ea6"
server_key_iv: "f4ff57a36e6520cd1efeb1033141a9e9"
server_key_version: "encryptionkey@0"
I20260811 02:57:52.959949 21170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:52.977849 21170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:52.980254 21170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:52.980417 21170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:52.981006 21170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:52.981779 21170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:52.981824 21170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:52.981884 21170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:52.981912 21170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:52.990326 21170 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:33265
I20260811 02:57:52.990368 21415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:33265 every 8 connection(s)
I20260811 02:57:52.991154 21170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:52.994953 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 21170
I20260811 02:57:52.995065 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:57:52.995380 9005 external_mini_cluster.cc:1428] Setting key 1552f92cf7860be880d1b86adc0eb48c
Aug 11 02:57:52 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:57:53.004266 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:52.992825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:51555 (local address 127.8.203.126:43163)
0811 02:57:52.992954 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:52.992958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:52.993806 (+ 848us) server_negotiation.cc:407] Connection header received
0811 02:57:52.995110 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:52.995113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:52.995160 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:52.995229 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:52.996653 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:52.997181 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:52.998329 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:52.998558 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:52.998645 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.001051 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.001054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.001056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.001079 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.002376 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.002855 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.002858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.002859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.002941 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.003259 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.003263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.003264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.003435 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.003518 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.003987 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.004107 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for oryx/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:57:53.004980 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:52.993183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:51555)
0811 02:57:52.993653 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:52.993695 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:52.994871 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:52.995426 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:52.995437 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:52.995886 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:52.996495 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:52.996514 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:52.997348 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:52.997351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:52.998197 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:52.998206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:52.998701 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:52.998705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:52.998860 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:52.999391 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:52.999407 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.000894 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.002491 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.002497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.002512 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.002756 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.003031 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.003034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.003036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.003166 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.003541 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.003547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.003761 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.003920 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.004217 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":154,"threads_started":1}
I20260811 02:57:53.006187 21416 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43163
I20260811 02:57:53.006415 21416 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:53.006865 21416 heartbeater.cc:511] Master 127.8.203.126:43163 requested a full tablet report, sending...
I20260811 02:57:53.008255 21056 ts_manager.cc:194] Registered new tserver with Master: bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.66@KRBTEST.COM" created.
I20260811 02:57:53.009682 21056 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:51555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab.
Entry for principal oryx/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab.
I20260811 02:57:53.022346 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43163
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab
--principal=oryx/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:53.135680 21426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:53.135937 21426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:53.135979 21426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:53.139957 21426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:53.140007 21426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:53.140061 21426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:53.144996 21426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab
--principal=oryx/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:53.145931 21426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:53.146855 21426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:53.156145 21433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.156126 21432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:53.156195 21426 server_base.cc:1034] running on GCE node
W20260811 02:57:53.156147 21435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:53.156826 21426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:53.157475 21426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:53.158668 21426 hybrid_clock.cc:648] HybridClock initialized: now 1786417073158639 us; error 44 us; skew 500 ppm
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:53.161233 21426 init.cc:377] Logged in from keytab as oryx/127.8.203.66@KRBTEST.COM (short username oryx)
I20260811 02:57:53.162104 21426 webserver.cc:460] Webserver started at http://127.8.203.66:43697/ using document root <none> and password file <none>
I20260811 02:57:53.162649 21426 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:53.162709 21426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:53.162885 21426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:53.164901 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "b9d192581a374759a177fd9e50877f1a"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "c5ba21a12ba1220aaacbbf5a5bce76c2"
server_key_iv: "c84b0a0d61929ddbc16a41e2219edbc9"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.165446 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "b9d192581a374759a177fd9e50877f1a"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "c5ba21a12ba1220aaacbbf5a5bce76c2"
server_key_iv: "c84b0a0d61929ddbc16a41e2219edbc9"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.168885 21426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:53.171139 21446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.172273 21426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:57:53.172374 21426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "b9d192581a374759a177fd9e50877f1a"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "c5ba21a12ba1220aaacbbf5a5bce76c2"
server_key_iv: "c84b0a0d61929ddbc16a41e2219edbc9"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.172452 21426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:53.191884 21426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:53.194391 21426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:53.194525 21426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:53.194998 21426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:53.195724 21426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:53.195767 21426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.195803 21426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:53.195823 21426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.204156 21426 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:35499
I20260811 02:57:53.204185 21657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:35499 every 8 connection(s)
I20260811 02:57:53.204948 21426 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:53.208421 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 21426
I20260811 02:57:53.208513 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:57:53.208825 9005 external_mini_cluster.cc:1428] Setting key ef900b8b018b082080e1957071e45ce8
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:57:53.218489 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.206562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:44823 (local address 127.8.203.126:43163)
0811 02:57:53.206678 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.206681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.207420 (+ 739us) server_negotiation.cc:407] Connection header received
0811 02:57:53.208697 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.208701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.208760 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.208843 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.210460 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.210972 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.212107 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.212393 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.212478 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.214905 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.214908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.214910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.214934 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.216495 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.216978 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.216981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.216983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.217077 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.217396 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.217399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.217401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.217627 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.217730 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.218188 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.218314 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:57:53.219103 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.206852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:44823)
0811 02:57:53.207270 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.207310 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.208407 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.209115 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.209125 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.209640 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.210262 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.210284 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.211094 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.211098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.211952 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.211961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.212539 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.212543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.212733 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.213269 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.213308 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.214739 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.216606 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.216611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.216622 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.216875 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.217159 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.217161 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.217163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.217281 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.217723 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.217728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.217940 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.218078 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.218309 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 02:57:53.220161 21658 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43163
I20260811 02:57:53.220376 21658 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:53.220901 21658 heartbeater.cc:511] Master 127.8.203.126:43163 requested a full tablet report, sending...
I20260811 02:57:53.221869 21056 ts_manager.cc:194] Registered new tserver with Master: b9d192581a374759a177fd9e50877f1a (127.8.203.66:35499)
I20260811 02:57:53.222427 21056 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:44823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab.
Entry for principal oryx/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab.
I20260811 02:57:53.237715 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43163
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab
--principal=oryx/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:53.346906 21666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:53.347126 21666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:53.347164 21666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:53.351019 21666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:53.351064 21666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:53.351116 21666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:53.356211 21666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab
--principal=oryx/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:53.357059 21666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:53.357975 21666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:53.367015 21679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.367056 21676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.367018 21673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:53.367357 21666 server_base.cc:1034] running on GCE node
I20260811 02:57:53.367748 21666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:53.368307 21666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:53.369459 21666 hybrid_clock.cc:648] HybridClock initialized: now 1786417073369435 us; error 39 us; skew 500 ppm
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:53.372288 21666 init.cc:377] Logged in from keytab as oryx/127.8.203.67@KRBTEST.COM (short username oryx)
I20260811 02:57:53.373158 21666 webserver.cc:460] Webserver started at http://127.8.203.67:35289/ using document root <none> and password file <none>
I20260811 02:57:53.373723 21666 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:53.373797 21666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:53.374034 21666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:53.376345 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "f146d58cbc36798740db92d437d07f12"
server_key_iv: "fb4db2ec842dac4e338429bf1cbd0de3"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.376935 21666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "f146d58cbc36798740db92d437d07f12"
server_key_iv: "fb4db2ec842dac4e338429bf1cbd0de3"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.380532 21666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:57:53.382936 21689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.384094 21666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:57:53.384213 21666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "f146d58cbc36798740db92d437d07f12"
server_key_iv: "fb4db2ec842dac4e338429bf1cbd0de3"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.384298 21666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:53.398075 21666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:53.401104 21666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:53.401266 21666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:53.401824 21666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:53.402598 21666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:53.402649 21666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.402693 21666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:53.402717 21666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.411331 21666 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:42583
I20260811 02:57:53.411350 21906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:42583 every 8 connection(s)
I20260811 02:57:53.412186 21666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:53.413607 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 21666
I20260811 02:57:53.413712 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:57:53.413982 9005 external_mini_cluster.cc:1428] Setting key db6cffa6961c53ad6af1b8fe1dfa5538
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:57:53.425504 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.415265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:39465 (local address 127.8.203.126:43163)
0811 02:57:53.415429 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.415435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.415454 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:57:53.416370 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.416373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.416434 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.416515 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.417724 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.418257 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.419412 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.419636 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.419744 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.422213 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.422216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.422219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.422243 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.423428 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.423934 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.423938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.423941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.424031 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.424387 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.424391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.424393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.424577 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.424685 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.425165 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.425327 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:57:53.426204 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.414290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:39465)
0811 02:57:53.414772 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.414827 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.416143 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.416696 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.416703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.417041 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.417566 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.417582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.418395 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.418398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.419274 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.419285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.419794 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.419799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.419973 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.420524 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.420542 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.422032 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.423551 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.423558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.423572 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.423824 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.424129 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.424132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.424134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.424283 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.424693 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.424700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.424923 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.425095 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.425445 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":113,"threads_started":1}
I20260811 02:57:53.427294 21907 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43163
I20260811 02:57:53.427508 21907 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:53.427951 21907 heartbeater.cc:511] Master 127.8.203.126:43163 requested a full tablet report, sending...
I20260811 02:57:53.429013 21056 ts_manager.cc:194] Registered new tserver with Master: 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583)
I20260811 02:57:53.429641 21056 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:39465
I20260811 02:57:53.439257 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.203.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.125.keytab.
Entry for principal oryx/127.8.203.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.125.keytab.
I20260811 02:57:53.466063 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:33109
--webserver_interface=127.8.203.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.125.keytab
--principal=oryx/127.8.203.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43163,127.8.203.125:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:57:53.574442 21916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:53.574662 21916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:53.574699 21916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:53.578644 21916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:53.578694 21916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:53.578712 21916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:53.578727 21916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:53.578758 21916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:53.583807 21916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43163,127.8.203.125:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.125.keytab
--principal=oryx/127.8.203.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.203.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
W20260811 02:57:53.583981 21916 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.203.126:43163,127.8.203.125:33109'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:57:53.584740 21916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:53.585685 21916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:53.593614 21924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.593632 21925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:53.593755 21916 server_base.cc:1034] running on GCE node
W20260811 02:57:53.593916 21928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:53.594444 21916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:53.595475 21916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:53.596633 21916 hybrid_clock.cc:648] HybridClock initialized: now 1786417073596613 us; error 30 us; skew 500 ppm
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:53.599253 21916 init.cc:377] Logged in from keytab as oryx/127.8.203.125@KRBTEST.COM (short username oryx)
I20260811 02:57:53.600098 21916 webserver.cc:460] Webserver started at http://127.8.203.125:38435/ using document root <none> and password file <none>
I20260811 02:57:53.600607 21916 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:53.600672 21916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:53.600858 21916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:53.602838 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/instance:
uuid: "0a6b74a505854495aba97aba8566cc46"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "6df9179c14fb59902e562650cb4f8ba4"
server_key_iv: "3315837642bb59efc34de2203483d096"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.603369 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal/instance:
uuid: "0a6b74a505854495aba97aba8566cc46"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "6df9179c14fb59902e562650cb4f8ba4"
server_key_iv: "3315837642bb59efc34de2203483d096"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.606866 21916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:57:53.609064 21942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.610226 21916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:53.610335 21916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
uuid: "0a6b74a505854495aba97aba8566cc46"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "6df9179c14fb59902e562650cb4f8ba4"
server_key_iv: "3315837642bb59efc34de2203483d096"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.610423 21916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:53.626935 21916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:53.629946 21916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:53.630120 21916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:53.636886 21916 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.125:33109
I20260811 02:57:53.636905 22035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.125:33109 every 8 connection(s)
I20260811 02:57:53.637863 21916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:53.640563 22036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:53.641901 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 21916
I20260811 02:57:53.644730 22036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.203.125@KRBTEST.COM
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.125@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:57:53.655201 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.646431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.8.203.126:43163)
0811 02:57:53.646548 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.646553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.646879 (+ 326us) server_negotiation.cc:407] Connection header received
0811 02:57:53.646991 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.646994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.647041 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.647105 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.647752 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.648259 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.649142 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.649396 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.649486 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.651871 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.651875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.651879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.651901 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.653311 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.653772 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.653775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.653777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.653854 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.654168 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.654171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.654173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.654439 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.654541 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.654922 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.655041 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:57:53.656042 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.646377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:43848)
0811 02:57:53.646743 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.646781 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.646897 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.647176 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.647182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.647504 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.647622 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.647628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.648391 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.648394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.649020 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.649028 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.649535 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.649539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.649637 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.650109 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.650127 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.651726 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.653408 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.653416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.653426 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.653685 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.653947 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.653950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.653952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.654084 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.654535 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.654540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.654667 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.654859 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.655272 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":122,"thread_start_us":145,"threads_started":1}
I20260811 02:57:53.656042 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.643418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.8.203.125:33109)
0811 02:57:53.643835 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.643853 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.643890 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:57:53.644509 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.644526 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.644837 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.645138 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.645940 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.647294 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.648111 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.648571 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.648711 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.651384 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.651391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.651403 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.651427 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.653205 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.653714 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.653718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.653719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.653854 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.654140 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.654143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.654144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.654578 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.654730 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.654978 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.655188 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":285,"thread_start_us":138,"threads_started":1}
I20260811 02:57:53.657438 22036 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.125@KRBTEST.COM for oryx/127.8.203.125@KRBTEST.COM
I20260811 02:57:53.664417 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.657875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.0.0.1:43386)
0811 02:57:53.658020 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.658032 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.658103 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.658286 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.658288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.658415 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.658481 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.658486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.659412 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.659414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.659887 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.659893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.660232 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.660234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.660289 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.660669 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.660680 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.661773 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.663262 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.663264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.663266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.663442 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.663690 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.663692 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.663693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.663764 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.664074 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.664076 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.664150 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.664225 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.664307 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 02:57:53.664453 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.657930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.8.203.125:33109)
0811 02:57:53.658070 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.658073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.658084 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:57:53.658176 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.658178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.658209 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.658265 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.658559 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.659333 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.659965 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.660159 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.660222 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.661887 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.661890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.661892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.661912 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.663166 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.663521 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.663525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.663526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.663595 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.663833 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.663835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.663837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.663987 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.664058 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.664270 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.664358 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 02:57:53.666963 22036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Bootstrap starting.
I20260811 02:57:53.669063 22036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:53.669814 22036 log.cc:824] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:53.671582 22036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: No bootstrap required, opened a new log
I20260811 02:57:53.674402 22036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } }
I20260811 02:57:53.674585 22036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:53.674631 22036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a6b74a505854495aba97aba8566cc46, State: Initialized, Role: FOLLOWER
I20260811 02:57:53.675019 22036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } }
I20260811 02:57:53.675442 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } } }
I20260811 02:57:53.675549 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:53.676167 22036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:57:53.680629 22070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:53.680709 22070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:53.683212 21916 master_runner.cc:428] Detected that this master 0a6b74a505854495aba97aba8566cc46 is joining an existing cluster
I20260811 02:57:53.683274 21916 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.203.125:33109
I20260811 02:57:53.693470 22078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.685533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.0.0.1:43392)
0811 02:57:53.686315 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.686327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.686441 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.686660 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.686663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.686889 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.687032 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.687040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.687992 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.687996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.688754 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.688762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.689261 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.689265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.689403 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.689836 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.689851 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.690338 (+ 487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.692011 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.692014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.692016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.692193 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.692458 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.692461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.692463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.692540 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.692944 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.692947 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.693012 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.693095 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.693230 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":97,"threads_started":1}
I20260811 02:57:53.693473 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.685865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43392 (local address 127.8.203.125:33109)
0811 02:57:53.685989 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.685994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.686396 (+ 402us) server_negotiation.cc:407] Connection header received
0811 02:57:53.686534 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.686537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.686571 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.686630 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.687132 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.687857 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.688891 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.689162 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.689250 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.690477 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.690480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.690483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.690504 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.691908 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.692281 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.692284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.692286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.692376 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.692622 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.692625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.692627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.692831 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.692914 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.693145 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.693311 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":55}
I20260811 02:57:53.694020 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.685870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.8.203.126:43163)
0811 02:57:53.685986 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.685990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.686105 (+ 115us) server_negotiation.cc:407] Connection header received
0811 02:57:53.686207 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.686210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.686258 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.686332 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.686872 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.687514 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.688608 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.688966 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.689076 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.690596 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.690599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.690602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.690628 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.692167 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.692689 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.692692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.692694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.692787 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.693086 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.693090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.693091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.693332 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.693425 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.693746 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.693870 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:57:53.694569 22079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.685476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:43858)
0811 02:57:53.686022 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.686034 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.686132 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.686386 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.686390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.686595 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.686733 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.686741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.687624 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.687627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.688486 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.688496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.689096 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.689099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.689203 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.690081 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.690093 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.690464 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.692288 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.692291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.692293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.692564 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.692892 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.692895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.692896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.692987 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.693469 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.693472 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.693561 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.694271 (+ 710us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.694404 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":18,"thread_start_us":96,"threads_started":1}
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.126@KRBTEST.COM for oryx/127.8.203.125@KRBTEST.COM
I20260811 02:57:53.712636 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.700113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.8.203.125:33109)
0811 02:57:53.700202 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.700206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.700687 (+ 481us) server_negotiation.cc:407] Connection header received
0811 02:57:53.700935 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.700938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.700984 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.701029 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.701869 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.702601 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.703844 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.704176 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.704244 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.707655 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.707658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.707660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.707680 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.709081 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.711436 (+ 2355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.711440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.711442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.711519 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.711794 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.711797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.711799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.711964 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.712046 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.712392 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.712494 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 02:57:53.712653 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.700164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.0.0.1:43408)
0811 02:57:53.700567 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.700621 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.700780 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.701185 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.701191 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.701625 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.701746 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.701753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.702732 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.702736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.703715 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.703726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.704381 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.704384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.704491 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.705760 (+ 1269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.705776 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.707541 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.711095 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.711103 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.711108 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.711323 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.711610 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.711613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.711615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.711699 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.712073 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.712080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.712187 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.712302 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.712500 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":139,"threads_started":1}
I20260811 02:57:53.714239 21056 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.203.125:33109: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0a6b74a505854495aba97aba8566cc46"
member_type: NON_VOTER
last_known_addr {
host: "127.8.203.125"
port: 33109
}
attrs {
promote: true
}
}
dest_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e"
cas_config_opid_index: -1
I20260811 02:57:53.714986 21056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } }
I20260811 02:57:53.717550 21997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:53.718920 21997 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Refusing update from remote peer 1f2eba647abc4dcbb6aeb7ae4ee2587e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
WARNING: no policy specified for oryx/127.8.203.124@KRBTEST.COM; defaulting to no policy
I20260811 02:57:53.719429 22085 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0a6b74a505854495aba97aba8566cc46 (127.8.203.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } }
I20260811 02:57:53.719687 22084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.124@KRBTEST.COM" created.
I20260811 02:57:53.723541 22085 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.203.125:33109
I20260811 02:57:53.724064 21916 master.cc:537] Master@127.8.203.125:33109 shutting down...
I20260811 02:57:53.724527 22085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } }
I20260811 02:57:53.724694 22085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:53.727244 21997 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0a6b74a505854495aba97aba8566cc46 (127.8.203.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } }
I20260811 02:57:53.729311 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f2eba647abc4dcbb6aeb7ae4ee2587e. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } }
I20260811 02:57:53.729423 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:53.729512 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } }
I20260811 02:57:53.729578 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:53.729665 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } }
I20260811 02:57:53.729766 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:53.732847 22091 mvcc.cc:205] Tried to move back new op lower bound from 7317164330090045440 to 7317164329925099520. Current Snapshot: MvccSnapshot[applied={T|T < 7317164329960636416}]
I20260811 02:57:53.733075 21916 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 02:57:53.733330 21916 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 02:57:53.733420 21916 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.124.keytab.
Entry for principal oryx/127.8.203.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.124.keytab.
I20260811 02:57:53.740658 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:36081
--webserver_interface=127.8.203.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.124.keytab
--principal=oryx/127.8.203.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:57:53.748854 21916 master.cc:552] Master@127.8.203.125:33109 shutdown complete.
I20260811 02:57:53.748903 21916 master_runner.cc:305] Clearing existing system tablet
I20260811 02:57:53.749118 21916 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:53.749171 21916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:53.750300 21916 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:53.751152 22107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.751367 21916 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:53.751493 21916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:53.751550 21916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
uuid: "0a6b74a505854495aba97aba8566cc46"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "6df9179c14fb59902e562650cb4f8ba4"
server_key_iv: "3315837642bb59efc34de2203483d096"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.753046 21916 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:53.754974 21916 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:57:53.755149 21916 log.cc:1196] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 02:57:53.755419 21916 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Deleting consensus metadata
I20260811 02:57:53.755578 21916 master_runner.cc:315] Copying system tablet from 127.8.203.126:43163
I20260811 02:57:53.756770 21916 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: tablet copy: Beginning tablet copy session from remote peer at address 127.8.203.126:43163
I20260811 02:57:53.768615 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.757474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.8.203.126:43163)
0811 02:57:53.757602 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.757607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.761611 (+ 4004us) server_negotiation.cc:407] Connection header received
0811 02:57:53.761736 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.761739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.761793 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.761865 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.762338 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.763022 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.764143 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.764422 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.764516 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.765682 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.765686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.765689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.765715 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.767035 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.767450 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.767453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.767455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.767537 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.767797 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.767800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.767802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.767995 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.768081 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.768351 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.768458 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:57:53.768625 22114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.757381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:43860)
0811 02:57:53.761499 (+ 4118us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.761512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.761637 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.761908 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.761911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.762099 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.762209 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.762215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.763133 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.763137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.764023 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.764034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.764536 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.764540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.764635 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.765124 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.765137 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.765544 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.767146 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.767149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.767152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.767353 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.767625 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.767628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.767630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.767710 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.768091 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.768094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.768188 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.768274 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.768423 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4048,"thread_start_us":84,"threads_started":1}
I20260811 02:57:53.769544 21099 tablet_copy_service.cc:140] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0a6b74a505854495aba97aba8566cc46 ({username='oryx', principal='oryx/127.8.203.125@KRBTEST.COM'} at 127.0.0.1:43860)
I20260811 02:57:53.769716 21099 tablet_copy_service.cc:161] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0a6b74a505854495aba97aba8566cc46 at {username='oryx', principal='oryx/127.8.203.125@KRBTEST.COM'} at 127.0.0.1:43860: session id = 0a6b74a505854495aba97aba8566cc46-00000000000000000000000000000000
I20260811 02:57:53.771224 21099 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:57:53.772778 21916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:57:53.774753 21916 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: tablet copy: Starting download of 0 data blocks...
I20260811 02:57:53.774932 21916 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: tablet copy: Starting download of 1 WAL segments...
I20260811 02:57:53.776772 21916 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:57:53.778553 21099 tablet_copy_service.cc:342] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Request end of tablet copy session 0a6b74a505854495aba97aba8566cc46-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.203.125@KRBTEST.COM'} at 127.0.0.1:43860
I20260811 02:57:53.778644 21099 tablet_copy_service.cc:433] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: ending tablet copy session 0a6b74a505854495aba97aba8566cc46-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0a6b74a505854495aba97aba8566cc46
I20260811 02:57:53.785346 21916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:53.787057 22115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.787062 22118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.787150 22116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:53.787510 21916 server_base.cc:1034] running on GCE node
I20260811 02:57:53.787659 21916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:53.787926 21916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:53.789057 21916 hybrid_clock.cc:648] HybridClock initialized: now 1786417073789043 us; error 29 us; skew 500 ppm
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:53.790488 21916 init.cc:377] Logged in from keytab as oryx/127.8.203.125@KRBTEST.COM (short username oryx)
I20260811 02:57:53.790894 21916 webserver.cc:460] Webserver started at http://127.8.203.125:38679/ using document root <none> and password file <none>
I20260811 02:57:53.791064 21916 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:53.791098 21916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:53.792093 21916 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 02:57:53.792928 22130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.793146 21916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 02:57:53.793217 21916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
uuid: "0a6b74a505854495aba97aba8566cc46"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "6df9179c14fb59902e562650cb4f8ba4"
server_key_iv: "3315837642bb59efc34de2203483d096"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.793315 21916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:53.803714 21916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:53.805413 21916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:53.810002 21916 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.125:33109
I20260811 02:57:53.810088 22226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.125:33109 every 8 connection(s)
I20260811 02:57:53.810499 21916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
I20260811 02:57:53.811813 22229 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:53.812614 22229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Bootstrap starting.
I20260811 02:57:53.818431 22229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:53.818718 22229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Bootstrap complete.
I20260811 02:57:53.819168 22229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } }
I20260811 02:57:53.819271 22229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0a6b74a505854495aba97aba8566cc46, State: Initialized, Role: LEARNER
I20260811 02:57:53.819391 22229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } }
I20260811 02:57:53.819712 22236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } }
I20260811 02:57:53.819815 22229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:53.819890 22236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:53.822710 22263 catalog_manager.cc:1259] Loaded cluster ID: 63496041b446402299cdb4f2f54f28be
I20260811 02:57:53.822758 22263 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: loading cluster ID for follower catalog manager: success
I20260811 02:57:53.824227 22263 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: acquiring CA information for follower catalog manager: success
I20260811 02:57:53.825269 22263 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 02:57:53.894227 22103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:53.894472 22103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:53.894510 22103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:53.898448 22103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:57:53.898499 22103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:53.898517 22103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:53.898532 22103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:57:53.898551 22103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:57:53.903662 22103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.124.keytab
--principal=oryx/127.8.203.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:36081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.203.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:53.904582 22103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:53.905550 22103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:53.913594 22103 server_base.cc:1034] running on GCE node
W20260811 02:57:53.913560 22280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.913553 22286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:53.913561 22283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:53.914320 22103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:53.915347 22103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:53.916504 22103 hybrid_clock.cc:648] HybridClock initialized: now 1786417073916487 us; error 28 us; skew 500 ppm
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:53.919127 22103 init.cc:377] Logged in from keytab as oryx/127.8.203.124@KRBTEST.COM (short username oryx)
I20260811 02:57:53.920042 22103 webserver.cc:460] Webserver started at http://127.8.203.124:43621/ using document root <none> and password file <none>
I20260811 02:57:53.920648 22103 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:53.920717 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:53.920925 22103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:57:53.923033 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/instance:
uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "118ba632aa3ae2315df47b729f68b8af"
server_key_iv: "4f17def2dbeb38d84689ff1331a6ccfc"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.923561 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal/instance:
uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "118ba632aa3ae2315df47b729f68b8af"
server_key_iv: "4f17def2dbeb38d84689ff1331a6ccfc"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.927172 22103 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 02:57:53.929487 22297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:53.930651 22103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:57:53.930763 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "118ba632aa3ae2315df47b729f68b8af"
server_key_iv: "4f17def2dbeb38d84689ff1331a6ccfc"
server_key_version: "encryptionkey@0"
I20260811 02:57:53.930859 22103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:53.950587 22103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:53.953197 22103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:53.953384 22103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:53.960506 22103 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.124:36081
I20260811 02:57:53.960762 22385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.124:36081 every 8 connection(s)
I20260811 02:57:53.961535 22103 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:53.964417 22388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:53.968030 22388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:57:53.968541 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 22103
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.124@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.203.124@KRBTEST.COM
I20260811 02:57:53.980930 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.970085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.8.203.126:43163)
0811 02:57:53.970238 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.970243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.970600 (+ 357us) server_negotiation.cc:407] Connection header received
0811 02:57:53.971359 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.971362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.971413 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.971485 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.972281 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.972811 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.973945 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.974190 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.974291 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.977437 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.977441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.977443 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.977468 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.978963 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.979494 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.979497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.979498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.979575 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.979895 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.979898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.979900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.980101 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.980195 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.980661 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.980772 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 02:57:53.981738 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.969957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.8.203.124:36081)
0811 02:57:53.970361 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.970391 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.970431 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:57:53.971151 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.971170 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.971360 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.971527 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.972336 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.973164 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.974140 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.974646 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.974743 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.977419 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.977426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.977438 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.977463 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.979252 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.979697 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.979703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.979704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.979807 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.980083 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.980087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.980088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.980431 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.980586 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.980844 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.980984 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":284,"spinlock_wait_cycles":6272,"thread_start_us":107,"threads_started":1}
I20260811 02:57:53.981735 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.969956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:43874)
0811 02:57:53.970469 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.970505 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.971159 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.971553 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.971558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.971893 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.972138 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.972151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.972942 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.972945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.973824 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.973844 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.974345 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.974349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.974553 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.975112 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.975132 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.977282 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.979072 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.979079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.979087 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.979395 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.979667 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.979670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.979672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.979796 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.980204 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.980209 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.980433 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.980590 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.980895 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":4,"spinlock_wait_cycles":8960,"thread_start_us":165,"threads_started":1}
I20260811 02:57:53.982930 22388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.124@KRBTEST.COM for oryx/127.8.203.125@KRBTEST.COM
I20260811 02:57:53.990285 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.983345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.0.0.1:43422)
0811 02:57:53.983755 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.983768 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.983834 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.984083 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.984086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.984225 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.984303 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.984308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.985156 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.985159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.985714 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.985722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.986146 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.986149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.986239 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.986570 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.986580 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.987730 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.989141 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.989143 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.989145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.989356 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.989566 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.989568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.989569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.989634 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.989914 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.989916 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.989981 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.990049 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.990153 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346}
I20260811 02:57:53.990365 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.983511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.8.203.125:33109)
0811 02:57:53.983777 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.983782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.983800 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:57:53.983938 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.983940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.983981 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.984050 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.984420 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.985052 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.985822 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.986046 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.986130 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.987889 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.987892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.987894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.987916 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.989049 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.989424 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.989427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.989428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.989500 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.989696 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.989698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.989699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.989844 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.989918 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.990081 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.990218 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260811 02:57:53.991032 22388 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:57:53 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417073, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.124@KRBTEST.COM for oryx/127.8.203.124@KRBTEST.COM
I20260811 02:57:53.997962 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.991439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.0.0.1:59278)
0811 02:57:53.991520 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:53.991532 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:53.991595 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:53.991784 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:53.991786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:53.991901 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:53.991964 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.991968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.992870 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.992872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.993431 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:53.993440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.993971 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.993973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:53.994042 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.994431 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:53.994445 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:53.995472 (+ 1027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:53.996883 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.996885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.996887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.997042 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.997236 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:53.997238 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:53.997239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:53.997344 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:53.997615 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:53.997617 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:53.997683 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 02:57:53.997751 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 02:57:53.997845 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":1}
I20260811 02:57:53.998031 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:53.991514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.8.203.124:36081)
0811 02:57:53.991609 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:53.991612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:53.991625 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:53.991655 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:53.991658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:53.991692 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:53.991763 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:53.992064 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.992765 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.993536 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:53.993865 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:53.993967 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:53.995586 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:53.995589 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:53.995591 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:53.995611 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:53.996803 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.997109 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.997111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.997112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.997173 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:53.997408 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:53.997410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:53.997411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:53.997551 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:53.997618 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:53.997783 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 02:57:53.997898 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":41}
I20260811 02:57:54.000756 22388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Bootstrap starting.
I20260811 02:57:54.002991 22388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:54.003669 22388 log.cc:824] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:54.005474 22388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: No bootstrap required, opened a new log
I20260811 02:57:54.008335 22388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } }
I20260811 02:57:54.008533 22388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:54.008602 22388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b94b4fa4a484759bfb07a8a5278bc5e, State: Initialized, Role: FOLLOWER
I20260811 02:57:54.009014 22388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } }
I20260811 02:57:54.009477 22404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } } }
I20260811 02:57:54.009588 22404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:54.010177 22388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:54.011855 21416 heartbeater.cc:503] Master 127.8.203.126:43163 was elected leader, sending a full tablet report...
W20260811 02:57:54.014568 22423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:57:54.014642 22423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:57:54.017763 22103 master_runner.cc:428] Detected that this master 4b94b4fa4a484759bfb07a8a5278bc5e is joining an existing cluster
I20260811 02:57:54.017808 22103 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.203.124:36081
I20260811 02:57:54.029135 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.019667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:43888)
0811 02:57:54.019941 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.019955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.020042 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.020709 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.020712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.021091 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.021371 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.021381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.022430 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.022433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.024314 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.024322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.024965 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.024969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.025058 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.025519 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.025534 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.025922 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.027753 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.027756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.027757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.027949 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.028208 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.028209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.028211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.028278 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.028645 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.028647 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.028766 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.028858 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.028954 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":24,"spinlock_wait_cycles":896,"thread_start_us":100,"threads_started":1}
I20260811 02:57:54.029186 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.019712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.8.203.126:43163)
0811 02:57:54.020406 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.020411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.020425 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:54.020466 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.020469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.020514 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.020596 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.021492 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.022279 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.024443 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.024730 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.024800 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.026064 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.026068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.026071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.026098 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.027638 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.028037 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.028041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.028043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.028129 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.028354 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.028357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.028358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.028567 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.028659 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.028922 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.029034 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598}
I20260811 02:57:54.034901 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 21170
I20260811 02:57:54.041800 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.019714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43438 (local address 127.8.203.125:33109)
0811 02:57:54.019829 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.019833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.020462 (+ 629us) server_negotiation.cc:407] Connection header received
0811 02:57:54.020531 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.020534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.020579 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.020628 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.021435 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.022279 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.023681 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.024034 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.024104 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.038886 (+ 14782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.038889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.038892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.038918 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.040316 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.040724 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.040727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.040729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.040799 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.041024 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.041027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.041028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.041197 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.041256 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.041545 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.041658 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:57:54.042199 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.019917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.0.0.1:43438)
0811 02:57:54.020354 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.020365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.020450 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.020766 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.020768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.021126 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.021260 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.021268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.022429 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.022432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.023537 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.023549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.024267 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.024271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.024415 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.024939 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.024953 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.038767 (+ 13814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.040430 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.040433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.040435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.040635 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.040884 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.040886 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.040888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.040953 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.041322 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.041325 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.041406 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.041922 (+ 516us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.042054 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":34,"thread_start_us":93,"threads_started":1}
I20260811 02:57:54.043524 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.020106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.0.0.1:59290)
0811 02:57:54.020969 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.020982 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.021138 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.021460 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.021462 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.021860 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.022165 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.022175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.023135 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.023138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.024164 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.024175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.025109 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.025112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.025250 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.038389 (+ 13139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.038401 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.038672 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.041125 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.041128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.041131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.041431 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.041735 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.041738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.041740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.041834 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.042883 (+ 1049us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.042886 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.042982 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.043097 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.043245 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":6,"thread_start_us":86,"threads_started":1}
I20260811 02:57:54.044265 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.020811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.8.203.124:36081)
0811 02:57:54.020928 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.020932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.021084 (+ 152us) server_negotiation.cc:407] Connection header received
0811 02:57:54.021227 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.021230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.021266 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.021331 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.022288 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.023019 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.024282 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.024919 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.024989 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.038825 (+ 13836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.038830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.038833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.038860 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.040999 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.041542 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.041546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.041548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.041637 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.042493 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.042496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.042498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.042730 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.042840 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.043994 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.044121 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417072, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.126@KRBTEST.COM for oryx/127.8.203.124@KRBTEST.COM
I20260811 02:57:54.059620 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.032595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.0.0.1:59292)
0811 02:57:54.032706 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.032720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.032800 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.033375 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.033378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.033521 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.033600 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.033605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.034715 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.034718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.035431 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.035441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.036025 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.036029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.036114 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.037552 (+ 1438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.037564 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.055094 (+ 17530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.058050 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.058054 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.058056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.058266 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.058614 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.058617 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.058620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.058706 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.059088 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.059092 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.059191 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.059281 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.059451 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 02:57:54.059998 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.032673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.8.203.124:36081)
0811 02:57:54.033065 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.033099 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.033114 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:57:54.033174 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.033177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.033225 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.033336 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.033698 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.034590 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.035596 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.035895 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.036277 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.055555 (+ 19278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.055559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.055562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.055587 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.057919 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.058379 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.058382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.058383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.058509 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.058799 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.058802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.058803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.058984 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.059072 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.059361 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.059467 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":181,"threads_started":1}
I20260811 02:57:54.060943 21055 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.203.124:36081: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
member_type: NON_VOTER
last_known_addr {
host: "127.8.203.124"
port: 36081
}
attrs {
promote: true
}
}
dest_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e"
cas_config_opid_index: 5
I20260811 02:57:54.061668 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:33265
--local_ip_for_outbound_sockets=127.8.203.65
--tserver_master_addrs=127.8.203.126:43163
--webserver_port=40291
--webserver_interface=127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab
--principal=oryx/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:57:54.061741 21055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } }
I20260811 02:57:54.062911 22086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.063519 22339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:54.064358 22086 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.203.124:36081
I20260811 02:57:54.064657 22086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.064750 22086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:54.065196 22339 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Refusing update from remote peer 1f2eba647abc4dcbb6aeb7ae4ee2587e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 02:57:54.065654 22404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:54.065800 22404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } }
I20260811 02:57:54.066732 22404 leader_election.cc:288] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163), 0a6b74a505854495aba97aba8566cc46 (127.8.203.125:33109)
I20260811 02:57:54.066844 22103 master.cc:537] Master@127.8.203.124:36081 shutting down...
I20260811 02:57:54.067221 22086 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:54.069761 22185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a6b74a505854495aba97aba8566cc46" is_pre_election: true
I20260811 02:57:54.069954 22185 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Handling vote request from an unknown peer 4b94b4fa4a484759bfb07a8a5278bc5e
I20260811 02:57:54.070083 22185 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 4b94b4fa4a484759bfb07a8a5278bc5e for term 2 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
W20260811 02:57:54.070905 21029 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 02:57:54.071456 21078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" is_pre_election: true
I20260811 02:57:54.072003 22103 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:54.072296 22103 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:54.072412 22103 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: stopping tablet replica
I20260811 02:57:54.073415 22302 leader_election.cc:302] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b94b4fa4a484759bfb07a8a5278bc5e; no voters: 0a6b74a505854495aba97aba8566cc46, 1f2eba647abc4dcbb6aeb7ae4ee2587e
W20260811 02:57:54.073632 22302 raft_consensus.cc:2670] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260811 02:57:54.074735 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.066256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.0.0.1:43446)
0811 02:57:54.066424 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.066437 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.066517 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.067407 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.067410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.067429 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:54.067520 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.067526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.068557 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.068560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.072592 (+ 4032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.072603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.074313 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.074317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.074413 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.074460 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.074530 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.074609 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 02:57:54.075202 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.066930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.8.203.125:33109)
0811 02:57:54.067050 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.067054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.067068 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:54.067119 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.067122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.067181 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.067277 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:54.067643 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.068424 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.072728 (+ 4304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.073998 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.074100 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.074250 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.074992 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.075089 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:57:54.075664 22185 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Refusing update from remote peer 1f2eba647abc4dcbb6aeb7ae4ee2587e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 02:57:54.076586 22084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 02:57:54.078034 22185 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.079790 22236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f2eba647abc4dcbb6aeb7ae4ee2587e. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.079922 22236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:54.080006 22236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: NON_VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: true } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.080086 22236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:54.080636 22084 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: attempting to promote NON_VOTER 0a6b74a505854495aba97aba8566cc46 to VOTER
W20260811 02:57:54.080690 21029 connection.cc:535] client connection to 127.8.203.124:36081 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 02:57:54.080801 21029 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e -> Peer 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124:36081): Couldn't send request to peer 4b94b4fa4a484759bfb07a8a5278bc5e. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:54.081076 22084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } }
I20260811 02:57:54.082607 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.082301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.0.0.1:59294)
0811 02:57:54.082462 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.082532 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 02:57:54.083454 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.083233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.0.0.1:59300)
0811 02:57:54.083351 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.083395 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:36081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 02:57:54.083635 21029 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e -> Peer 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124:36081): Couldn't send request to peer 4b94b4fa4a484759bfb07a8a5278bc5e. Status: Network error: Client connection negotiation failed: client connection to 127.8.203.124:36081: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:54.085677 22185 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 LEARNER]: Refusing update from remote peer 1f2eba647abc4dcbb6aeb7ae4ee2587e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 02:57:54.086151 22447 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 02:57:54.088081 22086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0a6b74a505854495aba97aba8566cc46 (127.8.203.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.089543 22086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.089643 22086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:54.089748 22086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.089816 22086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:54.089920 22185 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0a6b74a505854495aba97aba8566cc46 (127.8.203.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.092756 22103 master.cc:552] Master@127.8.203.124:36081 shutdown complete.
I20260811 02:57:54.092818 22103 master_runner.cc:305] Clearing existing system tablet
I20260811 02:57:54.093052 22103 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:54.093098 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:54.094286 22450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.094386 22103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:54.094503 22450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:54.095319 22454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.095525 22103 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:54.095769 22103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:54.095862 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "118ba632aa3ae2315df47b729f68b8af"
server_key_iv: "4f17def2dbeb38d84689ff1331a6ccfc"
server_key_version: "encryptionkey@0"
I20260811 02:57:54.098296 22103 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:54.100674 22103 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:57:54.100840 22103 log.cc:1196] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 02:57:54.101151 22103 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Deleting consensus metadata
I20260811 02:57:54.101380 22103 master_runner.cc:315] Copying system tablet from 127.8.203.126:43163
I20260811 02:57:54.102550 22103 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: tablet copy: Beginning tablet copy session from remote peer at address 127.8.203.126:43163
I20260811 02:57:54.112430 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.103235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.8.203.126:43163)
0811 02:57:54.103357 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.103362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.103539 (+ 177us) server_negotiation.cc:407] Connection header received
0811 02:57:54.103658 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.103662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.103713 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.103807 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.104335 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.105140 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.106394 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.106729 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.106854 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.108167 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.108171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.108174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.108202 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.110479 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.111025 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.111029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.111031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.111126 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.111425 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.111428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.111430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.111764 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.111872 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.112164 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.112282 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:57:54.112571 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.103138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:43896)
0811 02:57:54.103441 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.103456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.103573 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.103852 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.103855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.104075 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.104214 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.104221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.105265 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.105269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.106255 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.106268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.106874 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.106878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.106993 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.107571 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.107585 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.108012 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.110647 (+ 2635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.110650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.110652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.110914 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.111207 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.111210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.111212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.111303 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.111888 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.111891 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.111988 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.112092 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.112226 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":88,"threads_started":1}
I20260811 02:57:54.113175 21099 tablet_copy_service.cc:140] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4b94b4fa4a484759bfb07a8a5278bc5e ({username='oryx', principal='oryx/127.8.203.124@KRBTEST.COM'} at 127.0.0.1:43896)
I20260811 02:57:54.113272 21099 tablet_copy_service.cc:161] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4b94b4fa4a484759bfb07a8a5278bc5e at {username='oryx', principal='oryx/127.8.203.124@KRBTEST.COM'} at 127.0.0.1:43896: session id = 4b94b4fa4a484759bfb07a8a5278bc5e-00000000000000000000000000000000
I20260811 02:57:54.114403 21099 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:57:54.115783 22103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:57:54.118338 22103 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: tablet copy: Starting download of 0 data blocks...
I20260811 02:57:54.118599 22103 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: tablet copy: Starting download of 1 WAL segments...
I20260811 02:57:54.120193 22103 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:57:54.122064 21099 tablet_copy_service.cc:342] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Request end of tablet copy session 4b94b4fa4a484759bfb07a8a5278bc5e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.203.124@KRBTEST.COM'} at 127.0.0.1:43896
I20260811 02:57:54.122138 21099 tablet_copy_service.cc:433] P 1f2eba647abc4dcbb6aeb7ae4ee2587e: ending tablet copy session 4b94b4fa4a484759bfb07a8a5278bc5e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4b94b4fa4a484759bfb07a8a5278bc5e
I20260811 02:57:54.130061 22103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:54.131821 22466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:54.131891 22469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:54.132140 22471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:54.132159 22103 server_base.cc:1034] running on GCE node
I20260811 02:57:54.132366 22103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:54.132665 22103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:54.133805 22103 hybrid_clock.cc:648] HybridClock initialized: now 1786417074133796 us; error 34 us; skew 500 ppm
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:54.135375 22103 init.cc:377] Logged in from keytab as oryx/127.8.203.124@KRBTEST.COM (short username oryx)
I20260811 02:57:54.135842 22103 webserver.cc:460] Webserver started at http://127.8.203.124:35881/ using document root <none> and password file <none>
I20260811 02:57:54.136039 22103 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:54.136082 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:54.137399 22103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:54.138365 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.138599 22103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:54.138723 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "118ba632aa3ae2315df47b729f68b8af"
server_key_iv: "4f17def2dbeb38d84689ff1331a6ccfc"
server_key_version: "encryptionkey@0"
I20260811 02:57:54.138877 22103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:54.154539 22103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:54.156790 22103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:54.161729 22103 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.124:36081
I20260811 02:57:54.161763 22604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.124:36081 every 8 connection(s)
I20260811 02:57:54.162459 22103 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
I20260811 02:57:54.164466 22605 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:57:54.165548 22605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Bootstrap starting.
I20260811 02:57:54.176585 22605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:57:54.176879 22605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: Bootstrap complete.
I20260811 02:57:54.177461 22605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } }
I20260811 02:57:54.177600 22605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4b94b4fa4a484759bfb07a8a5278bc5e, State: Initialized, Role: LEARNER
I20260811 02:57:54.177695 22605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } }
I20260811 02:57:54.178045 22611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.178156 22605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:57:54.178260 22611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:54.181150 22635 catalog_manager.cc:1259] Loaded cluster ID: 63496041b446402299cdb4f2f54f28be
I20260811 02:57:54.181262 22635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: loading cluster ID for follower catalog manager: success
I20260811 02:57:54.183584 22635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: acquiring CA information for follower catalog manager: success
I20260811 02:57:54.184948 22635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 02:57:54.209498 22445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:54.209725 22445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:54.209769 22445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:54.213792 22445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:54.213845 22445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:54.213915 22445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:57:54.219014 22445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab
--principal=oryx/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:33265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=40291
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:54.220009 22445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:54.221038 22445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:57:54.224642 21658 heartbeater.cc:503] Master 127.8.203.126:43163 was elected leader, sending a full tablet report...
W20260811 02:57:54.230782 22647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:54.230805 22643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:54.230932 22445 server_base.cc:1034] running on GCE node
W20260811 02:57:54.231096 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:54.231480 22445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:54.232059 22445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:54.233217 22445 hybrid_clock.cc:648] HybridClock initialized: now 1786417074233201 us; error 25 us; skew 500 ppm
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:54.235894 22445 init.cc:377] Logged in from keytab as oryx/127.8.203.65@KRBTEST.COM (short username oryx)
I20260811 02:57:54.236788 22445 webserver.cc:460] Webserver started at http://127.8.203.65:40291/ using document root <none> and password file <none>
I20260811 02:57:54.237365 22445 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:54.237474 22445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:54.241245 22445 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:57:54.243278 22661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.244448 22445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:54.244567 22445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "bfd1d5297db24ff58c7db4a7df308b7d"
format_stamp: "Formatted at 2026-08-11 02:57:52 on dist-test-slave-gjw7"
server_key: "3f78d306ddac21c2aafb9240f6249ea6"
server_key_iv: "f4ff57a36e6520cd1efeb1033141a9e9"
server_key_version: "encryptionkey@0"
I20260811 02:57:54.245085 22445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:54.259519 22445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:54.262102 22445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:54.262274 22445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:54.262801 22445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:54.263595 22445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:54.263641 22445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.263691 22445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:54.263722 22445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.271946 22445 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:33265
I20260811 02:57:54.272008 22871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:33265 every 8 connection(s)
I20260811 02:57:54.272872 22445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:57:54.280121 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 22445
I20260811 02:57:54.280303 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 21426
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:57:54.291378 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.274580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:52369 (local address 127.8.203.126:43163)
0811 02:57:54.274726 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.274731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.275557 (+ 826us) server_negotiation.cc:407] Connection header received
0811 02:57:54.277155 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.277158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.277214 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.277284 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.278925 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.279735 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.281397 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.281661 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.281722 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.285420 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.285424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.285426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.285449 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.289108 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.289739 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.289743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.289745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.289833 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.290214 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.290217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.290219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.290427 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.290494 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.291099 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.291224 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:57:54.292891 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.274948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:52369)
0811 02:57:54.275412 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.275460 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.276885 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.277535 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.277544 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.278166 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.278760 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.278777 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.279927 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.279930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.281246 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.281257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.281885 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.281888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.282134 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.283546 (+ 1412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.283564 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.285241 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.289258 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.289264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.289278 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.289632 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.289939 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.289942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.289944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.290104 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.290576 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.290582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.290838 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.291589 (+ 751us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.291908 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":175,"spinlock_wait_cycles":19968,"thread_start_us":135,"threads_started":1}
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.125@KRBTEST.COM
I20260811 02:57:54.294428 22876 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43163
I20260811 02:57:54.294651 22876 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.295214 22876 heartbeater.cc:511] Master 127.8.203.126:43163 requested a full tablet report, sending...
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.124@KRBTEST.COM
I20260811 02:57:54.296983 21055 ts_manager.cc:194] Re-registered known tserver with Master: bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265)
I20260811 02:57:54.297416 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:35499
--local_ip_for_outbound_sockets=127.8.203.66
--tserver_master_addrs=127.8.203.126:43163
--webserver_port=43697
--webserver_interface=127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab
--principal=oryx/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:57:54.297753 21055 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:52369
I20260811 02:57:54.299811 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.274560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:35669 (local address 127.8.203.125:33109)
0811 02:57:54.274682 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.274686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.275558 (+ 872us) server_negotiation.cc:407] Connection header received
0811 02:57:54.277079 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.277082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.277141 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.277194 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.278977 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.279814 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.291280 (+ 11466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.291609 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.291688 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.294042 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.294046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.294049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.294077 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.297729 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.298516 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.298520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.298521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.298609 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.298936 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.298939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.298941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.299143 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.299222 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.299540 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.299657 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:57:54.300279 22883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.274948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.8.203.65:35669)
0811 02:57:54.275412 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.275461 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.276843 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.277535 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.277544 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.278156 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.278756 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.278775 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.290357 (+ 11582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.290361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.291139 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.291151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.291841 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.291845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.291941 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.292422 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.292435 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.293896 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.298181 (+ 4285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.298183 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.298185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.298401 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.298740 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.298742 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.298744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.298830 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.299280 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.299283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.299363 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.299993 (+ 630us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.300121 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":78,"spinlock_wait_cycles":20352,"thread_start_us":141,"threads_started":1}
I20260811 02:57:54.301043 22878 heartbeater.cc:347] Connected to a master server at 127.8.203.125:33109
I20260811 02:57:54.301134 22878 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.301365 22878 heartbeater.cc:511] Master 127.8.203.125:33109 requested a full tablet report, sending...
I20260811 02:57:54.302366 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.275913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.8.203.65:59245)
0811 02:57:54.276191 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.276202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.276835 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.277535 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.277544 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.278160 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.278800 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.278810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.279924 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.279928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.294069 (+ 14141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.294080 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.294774 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.294779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.294896 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.295367 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.295380 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.298079 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.300766 (+ 2687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.300770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.300772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.301025 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.301388 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.301391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.301393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.301523 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.301921 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.301925 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.302011 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.302095 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.302236 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":145,"spinlock_wait_cycles":15488,"thread_start_us":73,"threads_started":1}
I20260811 02:57:54.302577 22166 ts_manager.cc:194] Registered new tserver with Master: bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265)
I20260811 02:57:54.302876 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.276317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:59245 (local address 127.8.203.124:36081)
0811 02:57:54.276626 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.276642 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.276657 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:57:54.277099 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.277102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.277232 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.277596 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.278897 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.279712 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.294203 (+ 14491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.294587 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.294715 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.299004 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.299008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.299011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.299040 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.300664 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.301160 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.301164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.301166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.301268 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.301608 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.301612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.301614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.301837 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.301929 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.302602 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.302734 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
I20260811 02:57:54.303651 22874 heartbeater.cc:347] Connected to a master server at 127.8.203.124:36081
I20260811 02:57:54.303745 22874 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.303884 22874 heartbeater.cc:511] Master 127.8.203.124:36081 requested a full tablet report, sending...
I20260811 02:57:54.304955 22526 ts_manager.cc:194] Registered new tserver with Master: bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265)
W20260811 02:57:54.408402 22890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:54.408617 22890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:54.408655 22890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:54.412542 22890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:54.412596 22890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:54.412671 22890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:57:54.417745 22890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab
--principal=oryx/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:35499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=43697
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:54.418668 22890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:54.419595 22890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:54.428958 22904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:54.428963 22902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:54.428963 22908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:54.429143 22890 server_base.cc:1034] running on GCE node
I20260811 02:57:54.429533 22890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:54.430105 22890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:54.431260 22890 hybrid_clock.cc:648] HybridClock initialized: now 1786417074431232 us; error 38 us; skew 500 ppm
I20260811 02:57:54.431493 21907 heartbeater.cc:503] Master 127.8.203.126:43163 was elected leader, sending a full tablet report...
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:54.434134 22890 init.cc:377] Logged in from keytab as oryx/127.8.203.66@KRBTEST.COM (short username oryx)
I20260811 02:57:54.434911 22890 webserver.cc:460] Webserver started at http://127.8.203.66:43697/ using document root <none> and password file <none>
I20260811 02:57:54.435416 22890 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:54.435488 22890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:54.439277 22890 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 02:57:54.441363 22929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.442521 22890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:54.442622 22890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "b9d192581a374759a177fd9e50877f1a"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "c5ba21a12ba1220aaacbbf5a5bce76c2"
server_key_iv: "c84b0a0d61929ddbc16a41e2219edbc9"
server_key_version: "encryptionkey@0"
I20260811 02:57:54.443121 22890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:54.455528 22890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:54.458495 22890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:54.458657 22890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:54.459194 22890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:54.460000 22890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:54.460054 22890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.460106 22890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:54.460134 22890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.464689 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.460834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.0.0.1:59316)
0811 02:57:54.460955 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.460969 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.461056 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.461377 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.461380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.461399 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:54.461497 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.461503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.462242 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.462245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.463393 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.463407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.464296 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.464300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.464396 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.464419 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.464480 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.464560 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 02:57:54.464689 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.460979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59316 (local address 127.8.203.124:36081)
0811 02:57:54.461133 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.461138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.461151 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:54.461203 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.461205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.461256 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.461361 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:54.461615 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.462125 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.463548 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.464181 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.464295 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.464448 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.464490 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.464565 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:57:54.465546 22550 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 02:57:54.465963 22611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f2eba647abc4dcbb6aeb7ae4ee2587e. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.466125 22611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: This master's current role is: LEARNER
I20260811 02:57:54.470862 22890 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:35499
I20260811 02:57:54.470983 23125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:35499 every 8 connection(s)
I20260811 02:57:54.471868 22890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:57:54.474370 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 22890
I20260811 02:57:54.474540 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 21666
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:57:54.490466 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.473664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:50223 (local address 127.8.203.126:43163)
0811 02:57:54.473786 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.473789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.478047 (+ 4258us) server_negotiation.cc:407] Connection header received
0811 02:57:54.479320 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.479324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.479381 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.479424 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.481163 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.481718 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.483226 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.483504 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.483580 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.487031 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.487035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.487038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.487067 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.488414 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.488913 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.488916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.488918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.488993 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.489310 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.489313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.489315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.489535 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.489630 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.490180 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.490301 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:57:54.491693 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.474511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:50223)
0811 02:57:54.477913 (+ 3402us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.477963 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.479113 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.479768 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.479776 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.480347 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.481029 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.481045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.481842 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.481845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.483108 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.483119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.483777 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.483780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.484017 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.484575 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.484595 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.486839 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.488524 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.488529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.488545 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.488820 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.489067 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.489069 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.489071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.489208 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.489682 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.489689 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.489924 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.490086 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.490411 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3174,"thread_start_us":215,"threads_started":1}
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.124@KRBTEST.COM
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.125@KRBTEST.COM
I20260811 02:57:54.496438 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.487158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:48835 (local address 127.8.203.124:36081)
0811 02:57:54.487344 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.487350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.487368 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:57:54.487415 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.487419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.487479 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.487576 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.488491 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.489460 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.490711 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.491041 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.491140 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.493337 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.493340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.493343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.493379 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.494855 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.495260 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.495263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.495265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.495341 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.495606 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.495609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.495611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.495799 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.495884 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.496168 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.496280 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 02:57:54.496810 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.475539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.8.203.66:48835)
0811 02:57:54.485479 (+ 9940us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.485491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.485563 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.487728 (+ 2165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.487732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.488152 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.488303 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.488314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.489688 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.489691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.490540 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.490552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.491260 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.491264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.491356 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.491833 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.491848 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.493187 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.494963 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.494967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.494969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.495164 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.495437 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.495440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.495442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.495521 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.495904 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.495908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.496010 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.496095 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.496226 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9878,"thread_start_us":2043,"threads_started":1}
I20260811 02:57:54.497795 23128 heartbeater.cc:347] Connected to a master server at 127.8.203.124:36081
I20260811 02:57:54.498050 23128 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.498243 23129 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43163
I20260811 02:57:54.498311 23129 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.498678 23128 heartbeater.cc:511] Master 127.8.203.124:36081 requested a full tablet report, sending...
I20260811 02:57:54.498678 23129 heartbeater.cc:511] Master 127.8.203.126:43163 requested a full tablet report, sending...
I20260811 02:57:54.499385 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.490507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:56357 (local address 127.8.203.125:33109)
0811 02:57:54.490646 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.490651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.490664 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:54.490706 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.490708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.490747 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.490812 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.491447 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.492173 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.493406 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.493761 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.493862 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.496097 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.496101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.496104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.496130 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.497723 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.498157 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.498160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.498162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.498241 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.498528 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.498532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.498533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.498733 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.498799 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.499118 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.499226 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 02:57:54.499897 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.477936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.8.203.66:56357)
0811 02:57:54.489531 (+ 11595us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.489545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.489622 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.490909 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.490912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.491130 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.491303 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.491312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.492308 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.492311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.493220 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.493232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.493971 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.493977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.494072 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.494541 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.494556 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.495947 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.497835 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.497838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.497840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.498054 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.498353 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.498355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.498357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.498437 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.498868 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.498870 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.498953 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.499599 (+ 646us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.499735 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11531,"thread_start_us":6629,"threads_started":1}
I20260811 02:57:54.499959 21055 ts_manager.cc:194] Re-registered known tserver with Master: b9d192581a374759a177fd9e50877f1a (127.8.203.66:35499)
I20260811 02:57:54.499980 22526 ts_manager.cc:194] Registered new tserver with Master: b9d192581a374759a177fd9e50877f1a (127.8.203.66:35499)
I20260811 02:57:54.500680 21055 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:50223
I20260811 02:57:54.500706 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:42583
--local_ip_for_outbound_sockets=127.8.203.67
--tserver_master_addrs=127.8.203.126:43163
--webserver_port=35289
--webserver_interface=127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab
--principal=oryx/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:57:54.501323 23130 heartbeater.cc:347] Connected to a master server at 127.8.203.125:33109
I20260811 02:57:54.501410 23130 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.501580 23130 heartbeater.cc:511] Master 127.8.203.125:33109 requested a full tablet report, sending...
I20260811 02:57:54.502265 22166 ts_manager.cc:194] Registered new tserver with Master: b9d192581a374759a177fd9e50877f1a (127.8.203.66:35499)
W20260811 02:57:54.616335 23137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:57:54.616587 23137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:57:54.616628 23137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:57:54.620657 23137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:57:54.620718 23137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:57:54.620776 23137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:57:54.626394 23137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab
--principal=oryx/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:42583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=35289
--tserver_master_addrs=127.8.203.126:43163,127.8.203.125:33109,127.8.203.124:36081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:57:54.627483 23137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:57:54.628535 23137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:57:54.638175 23144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:54.638213 23147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:57:54.638183 23145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:57:54.638561 23137 server_base.cc:1034] running on GCE node
I20260811 02:57:54.638981 23137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:57:54.639551 23137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:57:54.640707 23137 hybrid_clock.cc:648] HybridClock initialized: now 1786417074640681 us; error 36 us; skew 500 ppm
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:57:54.643308 23137 init.cc:377] Logged in from keytab as oryx/127.8.203.67@KRBTEST.COM (short username oryx)
I20260811 02:57:54.644187 23137 webserver.cc:460] Webserver started at http://127.8.203.67:35289/ using document root <none> and password file <none>
I20260811 02:57:54.644759 23137 fs_manager.cc:359] Metadata directory not provided
I20260811 02:57:54.644843 23137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:57:54.648993 23137 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:57:54.651145 23160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.652405 23137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:57:54.652513 23137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b"
format_stamp: "Formatted at 2026-08-11 02:57:53 on dist-test-slave-gjw7"
server_key: "f146d58cbc36798740db92d437d07f12"
server_key_iv: "fb4db2ec842dac4e338429bf1cbd0de3"
server_key_version: "encryptionkey@0"
I20260811 02:57:54.653064 23137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:57:54.669545 23137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:57:54.672348 23137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:57:54.672525 23137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:57:54.673110 23137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:57:54.674033 23137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:57:54.674093 23137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.674149 23137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:57:54.674175 23137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:57:54.683094 23137 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:42583
I20260811 02:57:54.683132 23359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:42583 every 8 connection(s)
I20260811 02:57:54.684088 23137 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:57:54.688251 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 23137
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.124@KRBTEST.COM
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
Aug 11 02:57:54 dist-test-slave-gjw7 krb5kdc[20971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417074, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.125@KRBTEST.COM
I20260811 02:57:54.702091 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.685901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:33091 (local address 127.8.203.124:36081)
0811 02:57:54.686023 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.686028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.687138 (+ 1110us) server_negotiation.cc:407] Connection header received
0811 02:57:54.688455 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.688458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.688511 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.688559 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.690543 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.691335 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.693567 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.693927 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.693996 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.697315 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.697320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.697323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.697348 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.699097 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.699694 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.699698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.699700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.699782 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.700383 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.700386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.700388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.700624 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.700716 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.701803 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.701927 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:57:54.702464 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.690990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.8.203.126:43163)
0811 02:57:54.692136 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.692141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.692379 (+ 238us) server_negotiation.cc:407] Connection header received
0811 02:57:54.692467 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.692470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.692514 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.692588 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.693581 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.694292 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.695379 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.695691 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.695787 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.697727 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.697730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.697733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.697759 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.700253 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.700739 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.700743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.700745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.700839 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.701143 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.701146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.701148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.701394 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.701521 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.702168 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.702297 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":70,"threads_started":1}
I20260811 02:57:54.702699 21135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.687629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:40735 (local address 127.8.203.126:43163)
0811 02:57:54.687741 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.687745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.687854 (+ 109us) server_negotiation.cc:407] Connection header received
0811 02:57:54.688446 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.688450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.688506 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.688559 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.690539 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.691330 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.693054 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.693436 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.693515 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.699424 (+ 5909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.699428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.699431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.699458 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.700958 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.701434 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.701438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.701439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.701526 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.701817 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.701820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.701822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.702034 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.702114 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.702450 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.702555 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 02:57:54.703541 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.685894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:42051 (local address 127.8.203.125:33109)
0811 02:57:54.686023 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.686028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.687141 (+ 1113us) server_negotiation.cc:407] Connection header received
0811 02:57:54.688451 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.688455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.688513 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.688564 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.690543 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.691252 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.694423 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.694742 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.694849 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.698417 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.698422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.698424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.698454 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.700622 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.701561 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.701565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.701567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.701647 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.702643 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.702646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.702648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.702846 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.702924 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.703277 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.703395 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":51}
I20260811 02:57:54.704262 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.686435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.8.203.67:33091)
0811 02:57:54.686976 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.687035 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.688193 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.688960 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.688969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.689642 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.690378 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.690394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.691475 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.691478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.693446 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.693456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.694525 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.694529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.694624 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.695090 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.695106 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.697114 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.699215 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.699221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.699235 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.699583 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.699892 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.699894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.699896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.700070 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.700771 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.700776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.701330 (+ 554us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.702479 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.703087 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":1128,"spinlock_wait_cycles":107392,"thread_start_us":138,"threads_started":1}
I20260811 02:57:54.704308 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.686435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.8.203.67:42051)
0811 02:57:54.686976 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.687034 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.688188 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.688959 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.688968 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.689655 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.690372 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.690394 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.691428 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.691432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.694250 (+ 2818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.694261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.695319 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.695323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.695422 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.696163 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.696177 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.698268 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.700869 (+ 2601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.700872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.700874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.701098 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.701747 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.701749 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.701751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.701827 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.702985 (+ 1158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.702987 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.703068 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.703158 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.703278 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":1599,"spinlock_wait_cycles":28672,"thread_start_us":146,"threads_started":1}
I20260811 02:57:54.705619 23362 heartbeater.cc:347] Connected to a master server at 127.8.203.124:36081
I20260811 02:57:54.705668 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.687494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:40735)
0811 02:57:54.687765 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.687778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.688198 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.688960 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.688969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.689601 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:57:54.690381 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.690394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.691475 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.691478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.692916 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.692928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.693705 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.693708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.693896 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.694928 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:57:54.694948 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:57:54.699275 (+ 4327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:57:54.701071 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.701074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.701075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.701322 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.701636 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:57:54.701639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:57:54.701641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:57:54.701719 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:57:54.702173 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:57:54.702176 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:57:54.702262 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.703982 (+ 1720us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.704145 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":267,"spinlock_wait_cycles":4992,"thread_start_us":90,"threads_started":1}
I20260811 02:57:54.705832 23362 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.706328 23362 heartbeater.cc:511] Master 127.8.203.124:36081 requested a full tablet report, sending...
I20260811 02:57:54.705623 23364 heartbeater.cc:347] Connected to a master server at 127.8.203.125:33109
I20260811 02:57:54.706888 23364 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.707034 23364 heartbeater.cc:511] Master 127.8.203.125:33109 requested a full tablet report, sending...
I20260811 02:57:54.707353 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.690610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.8.203.125:33109)
0811 02:57:54.692863 (+ 2253us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.692867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.692880 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:54.692914 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.692917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.692958 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.693967 (+ 1009us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.694775 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.695633 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.696893 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.697216 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.697316 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.698739 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.698743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.698746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.698770 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.702520 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.705896 (+ 3376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.705900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.705901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.705986 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.706282 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.706285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.706287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.706456 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.706567 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.707077 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.707198 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":2148,"thread_start_us":82,"threads_started":1}
I20260811 02:57:54.707770 22166 ts_manager.cc:194] Registered new tserver with Master: 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583)
I20260811 02:57:54.705581 21055 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:57:54.709143 22526 ts_manager.cc:194] Registered new tserver with Master: 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583)
W20260811 02:57:54.709518 21055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:54.711301 23363 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43163
I20260811 02:57:54.711390 23363 heartbeater.cc:464] Registering TS with master...
I20260811 02:57:54.711552 23363 heartbeater.cc:511] Master 127.8.203.126:43163 requested a full tablet report, sending...
I20260811 02:57:54.712452 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.691610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.8.203.124:36081)
0811 02:57:54.691889 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.691893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.692686 (+ 793us) server_negotiation.cc:407] Connection header received
0811 02:57:54.693213 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.693216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.693261 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.693334 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:54.699180 (+ 5846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.699882 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.706638 (+ 6756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.707171 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.707242 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.708921 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:54.708925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:54.708928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:54.708953 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:54.710626 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.711087 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.711091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.711092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.711182 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:54.711493 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:54.711497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:54.711498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:54.711771 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:54.711896 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.712194 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.712304 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
I20260811 02:57:54.712870 22084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: NON_VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: true } } }
I20260811 02:57:54.712996 22084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:54.713625 21056 ts_manager.cc:194] Re-registered known tserver with Master: 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583)
I20260811 02:57:54.714242 21056 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:40735
I20260811 02:57:54.716173 22084 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: attempting to promote NON_VOTER 4b94b4fa4a484759bfb07a8a5278bc5e to VOTER
I20260811 02:57:54.716449 22084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:54.717640 22185 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Refusing update from remote peer 1f2eba647abc4dcbb6aeb7ae4ee2587e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 02:57:54.718008 22548 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 LEARNER]: Refusing update from remote peer 1f2eba647abc4dcbb6aeb7ae4ee2587e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 02:57:54.718015 23392 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 02:57:54.718441 22084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 02:57:54.719666 23392 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.720242 22548 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.721338 22084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.721451 22084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:54.721612 23392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.721690 23392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:54.721900 22611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.721999 22611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:54.721971 22084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.722065 22084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:54.722293 22185 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.723609 23408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:54.723742 23408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:54.735590 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.730504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:33265 (local address 127.0.0.1:40694)
0811 02:57:54.730745 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.730757 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.730835 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.731592 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.731595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.731628 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:54.731809 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.731814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.733142 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.733146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.734236 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.734246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.735303 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.735307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.735375 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.735397 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.735434 (+ 37us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.735483 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":72,"spinlock_wait_cycles":7680,"thread_start_us":106,"threads_started":1}
I20260811 02:57:54.736249 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.730697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.8.203.65:33265)
0811 02:57:54.731086 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.731099 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.731125 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:57:54.731206 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.731212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.731443 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.731609 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:54.731946 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.733039 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.734372 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.735177 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.735765 (+ 588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.735934 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.735979 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.736064 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":107,"threads_started":1}
I20260811 02:57:54.736601 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.730026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42583 (local address 127.0.0.1:46474)
0811 02:57:54.730164 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.730179 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.730259 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.731592 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.731595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.731626 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:54.731806 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.731813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.734333 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.734345 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.735062 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.735072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.736229 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.736233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.736320 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.736342 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.736393 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.736481 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":38,"spinlock_wait_cycles":6272}
I20260811 02:57:54.736737 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.730853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:35499 (local address 127.0.0.1:47124)
0811 02:57:54.731825 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.731836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.731928 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.732438 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.732440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.732455 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:54.732628 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.732634 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.733995 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.733999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.735159 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.735171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.736406 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.736410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.736488 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.736509 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.736568 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.736631 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":24,"thread_start_us":92,"threads_started":1}
I20260811 02:57:54.737128 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.731024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.8.203.66:35499)
0811 02:57:54.731307 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.731368 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.732025 (+ 657us) server_negotiation.cc:407] Connection header received
0811 02:57:54.732108 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.732113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.732305 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.732464 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:54.732735 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.733876 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.735285 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.736089 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.736196 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.736345 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.736888 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.736990 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":81,"threads_started":1}
I20260811 02:57:54.739504 22747 tablet_service.cc:1467] Processing CreateTablet for tablet 67f4d805cf7545a39cb05a4bcefc96cc (DEFAULT_TABLE table=test-table [id=f01237c070cd4c8386a2f0bc6153e662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:54.739768 23005 tablet_service.cc:1467] Processing CreateTablet for tablet 67f4d805cf7545a39cb05a4bcefc96cc (DEFAULT_TABLE table=test-table [id=f01237c070cd4c8386a2f0bc6153e662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:54.740674 23005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f4d805cf7545a39cb05a4bcefc96cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:54.740674 22747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f4d805cf7545a39cb05a4bcefc96cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:54.745389 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.730173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.8.203.67:42583)
0811 02:57:54.731193 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.731203 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.731220 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:57:54.731276 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.731280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.731443 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.731609 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:54.733177 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.734227 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.735183 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.736109 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.736950 (+ 841us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.745014 (+ 8064us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.745071 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.745167 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":930,"thread_start_us":108,"threads_started":1}
I20260811 02:57:54.747881 23240 tablet_service.cc:1467] Processing CreateTablet for tablet 67f4d805cf7545a39cb05a4bcefc96cc (DEFAULT_TABLE table=test-table [id=f01237c070cd4c8386a2f0bc6153e662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:54.748854 23240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f4d805cf7545a39cb05a4bcefc96cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:54.749620 23418 tablet_bootstrap.cc:492] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Bootstrap starting.
I20260811 02:57:54.750630 23421 tablet_bootstrap.cc:492] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Bootstrap starting.
I20260811 02:57:54.752535 23418 tablet_bootstrap.cc:654] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:54.752609 23421 tablet_bootstrap.cc:654] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:54.753345 23421 log.cc:824] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:54.753582 23418 log.cc:824] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:54.755118 23421 tablet_bootstrap.cc:492] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: No bootstrap required, opened a new log
I20260811 02:57:54.755291 23421 ts_tablet_manager.cc:1397] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:57:54.755563 23418 tablet_bootstrap.cc:492] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: No bootstrap required, opened a new log
I20260811 02:57:54.755751 23418 ts_tablet_manager.cc:1397] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:57:54.757100 23426 tablet_bootstrap.cc:492] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Bootstrap starting.
I20260811 02:57:54.759428 23421 raft_consensus.cc:348] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.759697 23421 raft_consensus.cc:374] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:54.759759 23421 raft_consensus.cc:729] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9d192581a374759a177fd9e50877f1a, State: Initialized, Role: FOLLOWER
I20260811 02:57:54.759618 23418 raft_consensus.cc:348] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.759824 23418 raft_consensus.cc:374] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:54.759888 23418 raft_consensus.cc:729] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd1d5297db24ff58c7db4a7df308b7d, State: Initialized, Role: FOLLOWER
I20260811 02:57:54.760273 23418 consensus_queue.cc:260] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.760314 23421 consensus_queue.cc:260] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.760404 23426 tablet_bootstrap.cc:654] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:54.761211 23418 ts_tablet_manager.cc:1428] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:57:54.761189 23426 log.cc:824] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:57:54.761255 23421 ts_tablet_manager.cc:1428] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:57:54.761400 22876 heartbeater.cc:503] Master 127.8.203.126:43163 was elected leader, sending a full tablet report...
I20260811 02:57:54.763597 23129 heartbeater.cc:503] Master 127.8.203.126:43163 was elected leader, sending a full tablet report...
I20260811 02:57:54.766007 23426 tablet_bootstrap.cc:492] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: No bootstrap required, opened a new log
I20260811 02:57:54.766188 23426 ts_tablet_manager.cc:1397] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260811 02:57:54.769641 23426 raft_consensus.cc:348] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.769855 23426 raft_consensus.cc:374] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:54.769894 23426 raft_consensus.cc:729] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9d4b5c0a994c33b72ed3a0931e2e6b, State: Initialized, Role: FOLLOWER
I20260811 02:57:54.770251 23426 consensus_queue.cc:260] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.770838 23363 heartbeater.cc:503] Master 127.8.203.126:43163 was elected leader, sending a full tablet report...
I20260811 02:57:54.771124 23426 ts_tablet_manager.cc:1428] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260811 02:57:54.774794 22879 tablet.cc:2369] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:57:54.935422 23367 tablet.cc:2369] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:54.946643 23428 raft_consensus.cc:482] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:54.946842 23428 raft_consensus.cc:504] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.948076 23428 leader_election.cc:288] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583), bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265)
I20260811 02:57:54.952941 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.948418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:54407 (local address 127.8.203.67:42583)
0811 02:57:54.948601 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.948606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.948623 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:57:54.948764 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.948767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.948824 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.948911 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:54.949265 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.950105 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.951729 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.952321 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.952438 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.952491 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.952739 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.952808 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 02:57:54.952941 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.948510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:41363 (local address 127.8.203.65:33265)
0811 02:57:54.948644 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:54.948648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:54.948683 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:57:54.948787 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:54.948789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:54.948863 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:54.948933 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:54.949339 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.950105 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.951732 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.952321 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.952439 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.952491 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:54.952739 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 02:57:54.952807 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:57:54.952960 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.948364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:33265 (local address 127.8.203.66:41363)
0811 02:57:54.948631 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.948643 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.948712 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.948940 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.948942 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.948975 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:54.949214 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.949222 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.950214 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.950217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.951605 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.951617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.952435 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.952438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.952600 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.952650 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.952713 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.952805 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":98,"spinlock_wait_cycles":8064}
I20260811 02:57:54.952960 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:54.948341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42583 (local address 127.8.203.66:54407)
0811 02:57:54.948537 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:54.948552 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:54.948649 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:54.948939 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:54.948942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:54.948977 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:54.949149 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.949157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.950214 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.950217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.951605 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:54.951617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:54.952435 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:54.952438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:54.952604 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:54.952639 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:57:54.952713 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:57:54.952806 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":223,"spinlock_wait_cycles":10880}
I20260811 02:57:54.953573 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f4d805cf7545a39cb05a4bcefc96cc" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" is_pre_election: true
I20260811 02:57:54.953641 23272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f4d805cf7545a39cb05a4bcefc96cc" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" is_pre_election: true
I20260811 02:57:54.953873 22783 raft_consensus.cc:2454] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 0.
I20260811 02:57:54.953909 23272 raft_consensus.cc:2454] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 0.
I20260811 02:57:54.954344 22937 leader_election.cc:302] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9d192581a374759a177fd9e50877f1a, bfd1d5297db24ff58c7db4a7df308b7d; no voters:
I20260811 02:57:54.954608 23428 raft_consensus.cc:2781] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:54.954694 23428 raft_consensus.cc:482] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:54.954746 23428 raft_consensus.cc:3037] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:54.955772 23428 raft_consensus.cc:504] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.956081 23428 leader_election.cc:288] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 election: Requested vote from peers 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583), bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265)
I20260811 02:57:54.956424 23272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f4d805cf7545a39cb05a4bcefc96cc" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b"
I20260811 02:57:54.956455 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f4d805cf7545a39cb05a4bcefc96cc" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd1d5297db24ff58c7db4a7df308b7d"
I20260811 02:57:54.956544 23272 raft_consensus.cc:3037] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:54.956573 22783 raft_consensus.cc:3037] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:54.957618 23272 raft_consensus.cc:2454] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 1.
I20260811 02:57:54.957732 22783 raft_consensus.cc:2454] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 1.
I20260811 02:57:54.957904 22932 leader_election.cc:302] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f9d4b5c0a994c33b72ed3a0931e2e6b, b9d192581a374759a177fd9e50877f1a; no voters:
I20260811 02:57:54.958076 23428 raft_consensus.cc:2781] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:54.958254 23428 raft_consensus.cc:686] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 1 LEADER]: Becoming Leader. State: Replica: b9d192581a374759a177fd9e50877f1a, State: Running, Role: LEADER
I20260811 02:57:54.958734 23428 consensus_queue.cc:237] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:54.961983 21056 catalog_manager.cc:5495] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a reported cstate change: term changed from 0 to 1, leader changed from <none> to b9d192581a374759a177fd9e50877f1a (127.8.203.66). New cstate: current_term: 1 leader_uuid: "b9d192581a374759a177fd9e50877f1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } health_report { overall_health: HEALTHY } } }
W20260811 02:57:54.973882 23131 tablet.cc:2369] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:57:55.026252 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:55.022996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.8.203.66:35499)
0811 02:57:55.023127 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:55.023131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:55.023145 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:57:55.023241 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:55.023243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:55.023297 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:55.023387 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:55.023625 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.024116 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.024864 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.025068 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.025136 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:55.025413 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:55.025863 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:55.025971 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:55.026068 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 02:57:55.026118 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:57:55.032086 23272 raft_consensus.cc:1261] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Refusing update from remote peer b9d192581a374759a177fd9e50877f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:55.032087 22783 raft_consensus.cc:1261] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 1 FOLLOWER]: Refusing update from remote peer b9d192581a374759a177fd9e50877f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:55.032796 23428 consensus_queue.cc:1035] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:55.033131 23434 consensus_queue.cc:1035] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:55.042479 23440 mvcc.cc:205] Tried to move back new op lower bound from 7317164339324514304 to 7317164339033489408. Current Snapshot: MvccSnapshot[applied={T|T < 7317164339324514304}]
I20260811 02:57:55.055720 21055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:55.055891 21055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:55.059844 21055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f2eba647abc4dcbb6aeb7ae4ee2587e: Sending DeleteTablet for 3 replicas of tablet 67f4d805cf7545a39cb05a4bcefc96cc
I20260811 02:57:55.060644 22747 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f4d805cf7545a39cb05a4bcefc96cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:55 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:40694
I20260811 02:57:55.061038 23240 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f4d805cf7545a39cb05a4bcefc96cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:55 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:46474
I20260811 02:57:55.061596 23457 tablet_replica.cc:330] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: stopping tablet replica
I20260811 02:57:55.061606 23005 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f4d805cf7545a39cb05a4bcefc96cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:55 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:47124
I20260811 02:57:55.061964 23457 raft_consensus.cc:2229] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:55.062279 23457 raft_consensus.cc:2258] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:55.062711 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20997
I20260811 02:57:55.082320 23456 tablet_replica.cc:330] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: stopping tablet replica
I20260811 02:57:55.082773 23456 raft_consensus.cc:2229] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:55.083153 23456 raft_consensus.cc:2258] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:55.083284 23457 ts_tablet_manager.cc:1905] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:55.085911 23457 ts_tablet_manager.cc:1918] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:55.086026 23457 log.cc:1196] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/67f4d805cf7545a39cb05a4bcefc96cc
I20260811 02:57:55.086329 23457 ts_tablet_manager.cc:1939] T 67f4d805cf7545a39cb05a4bcefc96cc P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Deleting consensus metadata
I20260811 02:57:55.088991 23456 ts_tablet_manager.cc:1905] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:55.090394 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:55.062266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43468 (local address 127.8.203.125:33109)
0811 02:57:55.062389 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:55.062394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:55.062745 (+ 351us) server_negotiation.cc:407] Connection header received
0811 02:57:55.062954 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:55.062957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:55.062997 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:55.063040 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:55.076065 (+ 13025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.077162 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.078700 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.079141 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.079236 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:55.081510 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:55.081515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:55.081519 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:55.081555 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:55.083688 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.084360 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.084365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.084367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.084477 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.084864 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.084868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.084871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.085147 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:55.085278 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:55.090051 (+ 4773us) server_negotiation.cc:299] Negotiation successful
0811 02:57:55.090213 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:57:55.091804 23456 ts_tablet_manager.cc:1918] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:55.091913 23456 log.cc:1196] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/67f4d805cf7545a39cb05a4bcefc96cc
I20260811 02:57:55.092255 23456 ts_tablet_manager.cc:1939] T 67f4d805cf7545a39cb05a4bcefc96cc P bfd1d5297db24ff58c7db4a7df308b7d: Deleting consensus metadata
I20260811 02:57:55.098999 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:55.062330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.8.203.124:36081)
0811 02:57:55.062443 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:55.062447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:55.090835 (+ 28388us) server_negotiation.cc:407] Connection header received
0811 02:57:55.091088 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:55.091093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:55.091167 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:55.091231 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:55.092269 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.093061 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.094347 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.094571 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.094685 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:55.095950 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:55.095953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:55.095955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:55.095978 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:55.097413 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.097804 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.097807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.097809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.097896 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.098157 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.098160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.098162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.098338 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:55.098456 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:55.098691 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 02:57:55.098821 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:57:55.101536 23459 tablet_replica.cc:330] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: stopping tablet replica
I20260811 02:57:55.101871 23459 raft_consensus.cc:2229] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:55.102276 23459 raft_consensus.cc:2258] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 02:57:55.102967 22668 connection.cc:535] server connection from 127.0.0.1:40694 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 02:57:55.102977 23163 connection.cc:535] server connection from 127.0.0.1:46474 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:57:55.103827 23459 ts_tablet_manager.cc:1905] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:55.106743 23459 ts_tablet_manager.cc:1918] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:55.106851 23459 log.cc:1196] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/67f4d805cf7545a39cb05a4bcefc96cc
I20260811 02:57:55.107184 23459 ts_tablet_manager.cc:1939] T 67f4d805cf7545a39cb05a4bcefc96cc P b9d192581a374759a177fd9e50877f1a: Deleting consensus metadata
W20260811 02:57:55.108455 22933 connection.cc:441] server connection from 127.0.0.1:47124 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47124 (request call id 1) could send its response
I20260811 02:57:55.116894 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:55.106500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.8.203.125:33109)
0811 02:57:55.106636 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:55.106641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:55.107184 (+ 543us) server_negotiation.cc:407] Connection header received
0811 02:57:55.107441 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:55.107444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:55.107494 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:55.107544 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:55.109106 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.109810 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.110968 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.111210 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.111299 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:55.113429 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:55.113434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:55.113437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:55.113468 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:55.115236 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.115653 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.115657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.115658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.115748 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.116053 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.116056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.116058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.116237 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:55.116329 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:55.116623 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 02:57:55.116727 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:57:55.117527 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:55.107145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.8.203.124:36081)
0811 02:57:55.107260 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:55.107265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:55.108038 (+ 773us) server_negotiation.cc:407] Connection header received
0811 02:57:55.108215 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:55.108219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:55.108265 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:55.108317 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:57:55.109544 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.110148 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.111063 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:55.111406 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:55.111519 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:55.113486 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:57:55.113489 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:57:55.113492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:57:55.113517 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:57:55.114900 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.115286 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.115290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.115292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.115392 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:57:55.115729 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:57:55.115732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:57:55.115734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:57:55.115957 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:57:55.116069 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:55.117280 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0811 02:57:55.117423 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:57:56.045914 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.045365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:35295)
0811 02:57:56.045688 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.045790 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":81,"threads_started":1}
W20260811 02:57:56.046221 23363 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:43163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.048283 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.047684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:36445)
0811 02:57:56.048086 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.048173 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":65,"threads_started":1}
W20260811 02:57:56.048662 23129 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:43163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.049211 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.048600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:44319)
0811 02:57:56.048931 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.049109 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":66,"threads_started":1}
W20260811 02:57:56.049513 22876 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:43163 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.561007 23478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f2eba647abc4dcbb6aeb7ae4ee2587e)
I20260811 02:57:56.561241 23478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:56.561435 23479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f2eba647abc4dcbb6aeb7ae4ee2587e)
I20260811 02:57:56.561570 23479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:56.562065 23479 leader_election.cc:288] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163), 0a6b74a505854495aba97aba8566cc46 (127.8.203.125:33109)
I20260811 02:57:56.562367 23478 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163), 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124:36081)
I20260811 02:57:56.562820 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.562235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44352)
0811 02:57:56.562626 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.562717 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":78,"threads_started":1}
W20260811 02:57:56.563031 22499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.563402 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.562464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44362)
0811 02:57:56.562718 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.562790 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":57,"threads_started":1}
W20260811 02:57:56.563561 22136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.564448 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.564214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44380)
0811 02:57:56.564339 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.564387 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 02:57:56.564574 22136 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163): Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.564755 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.564522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44386)
0811 02:57:56.564644 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.564690 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 02:57:56.564880 22499 leader_election.cc:334] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163): Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.567252 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.562748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:36081 (local address 127.0.0.1:59362)
0811 02:57:56.563068 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.563080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:56.563162 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:56.563507 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:56.563511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:56.563530 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:56.563648 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:56.563655 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.565460 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.565464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:56.566250 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:56.566258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.566880 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.566883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:56.566973 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:56.566996 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:57:56.567063 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:57:56.567136 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":47,"threads_started":1}
I20260811 02:57:56.567312 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.563054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.8.203.124:36081)
0811 02:57:56.563244 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:56.563249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:56.563264 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:57:56.563313 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:56.563316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:56.563363 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:56.564617 (+ 1254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:56.564723 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.565351 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.566369 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.566767 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.566855 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:56.566960 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:56.567086 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 02:57:56.567187 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":133,"thread_start_us":59,"threads_started":1}
I20260811 02:57:56.567319 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.562513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33109 (local address 127.0.0.1:43498)
0811 02:57:56.562777 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.562790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:56.563006 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:56.563576 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:56.563579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:56.563598 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:56.563753 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:56.563761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.565470 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.565474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:56.566250 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:56.566258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.566863 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.566867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:56.567006 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:56.567025 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:57:56.567089 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 02:57:56.567186 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":39,"thread_start_us":69,"threads_started":1}
I20260811 02:57:56.567320 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.562473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.8.203.125:33109)
0811 02:57:56.563261 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:56.563264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:56.563274 (+ 10us) server_negotiation.cc:407] Connection header received
0811 02:57:56.563309 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:56.563312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:56.563355 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:56.563431 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:56.563952 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.565368 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.566370 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:56.566754 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:56.566853 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:56.566915 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:56.567117 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 02:57:56.567191 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":80,"threads_started":1}
I20260811 02:57:56.567795 22185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0a6b74a505854495aba97aba8566cc46" is_pre_election: true
I20260811 02:57:56.567797 22548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6b74a505854495aba97aba8566cc46" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" is_pre_election: true
I20260811 02:57:56.567905 22185 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b94b4fa4a484759bfb07a8a5278bc5e in term 1.
I20260811 02:57:56.567973 22548 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a6b74a505854495aba97aba8566cc46 in term 1.
I20260811 02:57:56.568153 22496 leader_election.cc:302] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6b74a505854495aba97aba8566cc46, 4b94b4fa4a484759bfb07a8a5278bc5e; no voters: 1f2eba647abc4dcbb6aeb7ae4ee2587e
I20260811 02:57:56.568279 22136 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6b74a505854495aba97aba8566cc46, 4b94b4fa4a484759bfb07a8a5278bc5e; no voters: 1f2eba647abc4dcbb6aeb7ae4ee2587e
I20260811 02:57:56.568369 23479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 02:57:56.568425 23479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f2eba647abc4dcbb6aeb7ae4ee2587e)
I20260811 02:57:56.568470 23479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:56.568609 23478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 02:57:56.568657 23478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f2eba647abc4dcbb6aeb7ae4ee2587e)
I20260811 02:57:56.568683 23478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:57:56.569523 23479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:56.569660 23478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:56.569835 23479 leader_election.cc:288] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 election: Requested vote from peers 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163), 0a6b74a505854495aba97aba8566cc46 (127.8.203.125:33109)
I20260811 02:57:56.570014 23478 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 2 election: Requested vote from peers 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163), 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124:36081)
I20260811 02:57:56.570125 22185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0a6b74a505854495aba97aba8566cc46"
I20260811 02:57:56.570217 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.569937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44398)
0811 02:57:56.570109 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.570159 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 02:57:56.570250 22185 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b94b4fa4a484759bfb07a8a5278bc5e in current term 2: Already voted for candidate 0a6b74a505854495aba97aba8566cc46 in this term.
I20260811 02:57:56.570441 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.570212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44414)
0811 02:57:56.570322 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.570382 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 02:57:56.570608 22548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6b74a505854495aba97aba8566cc46" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
I20260811 02:57:56.570751 22548 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a6b74a505854495aba97aba8566cc46 in current term 2: Already voted for candidate 4b94b4fa4a484759bfb07a8a5278bc5e in this term.
I20260811 02:57:56.570825 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.570621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44416)
0811 02:57:56.570702 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.570747 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 02:57:56.570914 22499 leader_election.cc:334] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163): Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.570968 22499 leader_election.cc:302] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b94b4fa4a484759bfb07a8a5278bc5e; no voters: 0a6b74a505854495aba97aba8566cc46, 1f2eba647abc4dcbb6aeb7ae4ee2587e
I20260811 02:57:56.571146 23479 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 02:57:56.571388 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:56.570801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44432)
0811 02:57:56.571286 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:56.571333 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 02:57:56.571480 22136 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163): Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:56.571581 22136 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a6b74a505854495aba97aba8566cc46; no voters: 1f2eba647abc4dcbb6aeb7ae4ee2587e, 4b94b4fa4a484759bfb07a8a5278bc5e
I20260811 02:57:56.571710 23478 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 02:57:57.047194 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:57.046718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:51867)
0811 02:57:57.047021 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:57.047096 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":106,"threads_started":1}
I20260811 02:57:57.049610 23496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:57.049194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:47769)
0811 02:57:57.049427 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:57.049491 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":59,"threads_started":1}
I20260811 02:57:57.050384 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:57.050010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:58217)
0811 02:57:57.050230 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:57.050288 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260811 02:57:58.048244 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.047807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:54709)
0811 02:57:58.048083 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.048144 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":83,"threads_started":1}
I20260811 02:57:58.050570 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.050204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:42103)
0811 02:57:58.050417 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.050476 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260811 02:57:58.051321 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.050929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:41129)
0811 02:57:58.051176 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.051231 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":64,"threads_started":1}
I20260811 02:57:58.345006 23503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:58.345127 23503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:58.345544 23503 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163), 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124:36081)
I20260811 02:57:58.346105 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.345722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44656)
0811 02:57:58.345949 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.346014 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":61,"threads_started":1}
I20260811 02:57:58.346242 22548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6b74a505854495aba97aba8566cc46" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" is_pre_election: true
I20260811 02:57:58.346396 22548 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a6b74a505854495aba97aba8566cc46 in term 2.
I20260811 02:57:58.346881 22136 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a6b74a505854495aba97aba8566cc46, 4b94b4fa4a484759bfb07a8a5278bc5e; no voters:
I20260811 02:57:58.347007 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.346755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44660)
0811 02:57:58.346864 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.346918 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 02:57:58.347071 23503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260811 02:57:58.347110 22136 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163): Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:58.347127 23503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:58.347211 23503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:58.348239 23503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:58.348551 23503 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 3 election: Requested vote from peers 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163), 4b94b4fa4a484759bfb07a8a5278bc5e (127.8.203.124:36081)
I20260811 02:57:58.348863 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.348650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44662)
0811 02:57:58.348734 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.348803 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 02:57:58.349063 22548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a6b74a505854495aba97aba8566cc46" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e"
I20260811 02:57:58.349179 22548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:57:58.349537 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.349353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44664)
0811 02:57:58.349431 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.349472 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 02:57:58.349666 22136 leader_election.cc:334] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163): Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
I20260811 02:57:58.350239 22548 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a6b74a505854495aba97aba8566cc46 in term 3.
I20260811 02:57:58.350487 22136 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a6b74a505854495aba97aba8566cc46, 4b94b4fa4a484759bfb07a8a5278bc5e; no voters: 1f2eba647abc4dcbb6aeb7ae4ee2587e
I20260811 02:57:58.350626 23503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:57:58.350805 23503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [term 3 LEADER]: Becoming Leader. State: Replica: 0a6b74a505854495aba97aba8566cc46, State: Running, Role: LEADER
I20260811 02:57:58.351051 23503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } }
I20260811 02:57:58.352000 23516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6b74a505854495aba97aba8566cc46. Latest consensus state: current_term: 3 leader_uuid: "0a6b74a505854495aba97aba8566cc46" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:58.352111 23516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:58.352408 23518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:57:58.354604 23518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:57:58.354982 23518 catalog_manager.cc:1259] Loaded cluster ID: 63496041b446402299cdb4f2f54f28be
I20260811 02:57:58.355026 23518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:57:58.355559 23518 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:57:58.355952 23518 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Loaded TSK: 0
I20260811 02:57:58.356653 23518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:57:58.395901 22163 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:57:58.398877 22163 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:57:58.401654 22548 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [term 3 FOLLOWER]: Refusing update from remote peer 0a6b74a505854495aba97aba8566cc46: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 02:57:58.401674 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.401461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44680)
0811 02:57:58.401568 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.401612 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 02:57:58.402248 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.402045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.0.0.1:44692)
0811 02:57:58.402139 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.402187 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 02:57:58.402272 23516 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 02:57:58.402436 22136 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 -> Peer 1f2eba647abc4dcbb6aeb7ae4ee2587e (127.8.203.126:43163): Couldn't send request to peer 1f2eba647abc4dcbb6aeb7ae4ee2587e. Status: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:57:58.403650 23520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6b74a505854495aba97aba8566cc46. Latest consensus state: current_term: 3 leader_uuid: "0a6b74a505854495aba97aba8566cc46" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:58.403776 23520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:58.404731 23503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0a6b74a505854495aba97aba8566cc46" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:58.404834 23503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46 [sys.catalog]: This master's current role is: LEADER
I20260811 02:57:58.405498 23520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0a6b74a505854495aba97aba8566cc46" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f2eba647abc4dcbb6aeb7ae4ee2587e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43163 } } peers { permanent_uuid: "0a6b74a505854495aba97aba8566cc46" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33109 } attrs { promote: false } } peers { permanent_uuid: "4b94b4fa4a484759bfb07a8a5278bc5e" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 36081 } attrs { promote: false } } }
I20260811 02:57:58.405620 23520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b94b4fa4a484759bfb07a8a5278bc5e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:57:58.413618 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.410017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:33265 (local address 127.0.0.1:40698)
0811 02:57:58.410127 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.410140 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:58.410231 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:58.410688 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:58.410691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:58.410704 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:58.410797 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:58.410803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.411740 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.411743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:58.412568 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:58.412578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.413228 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.413231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:58.413344 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:58.413367 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:57:58.413442 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:57:58.413511 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 02:57:58.413786 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.410128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.8.203.65:33265)
0811 02:57:58.410429 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:58.410435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:58.410448 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:58.410520 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:58.410524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:58.410576 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:58.410683 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:58.410931 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.411636 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.412690 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.413120 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.413211 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:58.413280 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:58.413585 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 02:57:58.413660 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":98,"threads_started":1}
I20260811 02:57:58.414309 22747 tablet_service.cc:1467] Processing CreateTablet for tablet 251176ddb9d845bea8e40d24d13be928 (DEFAULT_TABLE table=test-table [id=3cc4b34cca114648bfe996e02aaa0a51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:58.414603 22747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 251176ddb9d845bea8e40d24d13be928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:58.415014 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.410261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42583 (local address 127.0.0.1:46486)
0811 02:57:58.410494 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.410506 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:58.410581 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:58.411255 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:58.411258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:58.411276 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:58.411365 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:58.411371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.412337 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.412340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:58.413719 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:58.413730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.414637 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.414641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:58.414727 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:58.414749 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:57:58.414819 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:57:58.414894 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
I20260811 02:57:58.415174 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.410606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.8.203.67:42583)
0811 02:57:58.410928 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:58.410933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:58.410948 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:57:58.411002 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:58.411004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:58.411054 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:58.411136 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:58.411495 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.412213 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.413861 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.414437 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.414526 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:58.414583 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:58.414867 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 02:57:58.414946 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":72,"threads_started":1}
I20260811 02:57:58.415401 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.410880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:35499 (local address 127.0.0.1:47138)
0811 02:57:58.411460 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.411472 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:57:58.411557 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:57:58.411765 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:57:58.411768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:57:58.411786 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:57:58.411885 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:58.411891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.412839 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.412843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:58.414103 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:57:58.414112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.415024 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.415028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:57:58.415118 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:58.415142 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:57:58.415211 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 02:57:58.415278 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":87,"threads_started":1}
I20260811 02:57:58.415444 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.411213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.8.203.66:35499)
0811 02:57:58.411459 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:58.411463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:58.411525 (+ 62us) server_negotiation.cc:407] Connection header received
0811 02:57:58.411629 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:58.411632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:58.411679 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:58.411761 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:57:58.411987 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.412711 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.414229 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.414824 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.414911 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:58.414969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:58.415258 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 02:57:58.415331 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":76,"threads_started":1}
I20260811 02:57:58.415593 23240 tablet_service.cc:1467] Processing CreateTablet for tablet 251176ddb9d845bea8e40d24d13be928 (DEFAULT_TABLE table=test-table [id=3cc4b34cca114648bfe996e02aaa0a51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:58.415845 23240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 251176ddb9d845bea8e40d24d13be928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:58.417135 23537 tablet_bootstrap.cc:492] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d: Bootstrap starting.
I20260811 02:57:58.418012 23537 tablet_bootstrap.cc:654] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:58.418197 23538 tablet_bootstrap.cc:492] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Bootstrap starting.
I20260811 02:57:58.418915 23005 tablet_service.cc:1467] Processing CreateTablet for tablet 251176ddb9d845bea8e40d24d13be928 (DEFAULT_TABLE table=test-table [id=3cc4b34cca114648bfe996e02aaa0a51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:57:58.419106 23538 tablet_bootstrap.cc:654] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:58.419150 23537 tablet_bootstrap.cc:492] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d: No bootstrap required, opened a new log
I20260811 02:57:58.419171 23005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 251176ddb9d845bea8e40d24d13be928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:57:58.419236 23537 ts_tablet_manager.cc:1397] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:58.419585 23537 raft_consensus.cc:348] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.419697 23537 raft_consensus.cc:374] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:58.419730 23537 raft_consensus.cc:729] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd1d5297db24ff58c7db4a7df308b7d, State: Initialized, Role: FOLLOWER
I20260811 02:57:58.419863 23537 consensus_queue.cc:260] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.420215 23537 ts_tablet_manager.cc:1428] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:58.420315 23538 tablet_bootstrap.cc:492] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: No bootstrap required, opened a new log
I20260811 02:57:58.420392 23538 ts_tablet_manager.cc:1397] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:57:58.421041 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.420785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:45297)
0811 02:57:58.420908 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.420963 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 02:57:58.420778 23538 raft_consensus.cc:348] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.421382 23538 raft_consensus.cc:374] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:58.421424 23538 raft_consensus.cc:729] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9d4b5c0a994c33b72ed3a0931e2e6b, State: Initialized, Role: FOLLOWER
I20260811 02:57:58.421531 23538 consensus_queue.cc:260] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.421864 23538 ts_tablet_manager.cc:1428] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:58.422498 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.422274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:56361)
0811 02:57:58.422377 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.422434 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 02:57:58.422595 23546 tablet_bootstrap.cc:492] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: Bootstrap starting.
I20260811 02:57:58.423409 23546 tablet_bootstrap.cc:654] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: Neither blocks nor log segments found. Creating new log.
I20260811 02:57:58.423974 22878 heartbeater.cc:503] Master 127.8.203.125:33109 was elected leader, sending a full tablet report...
I20260811 02:57:58.424472 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.424257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:52545)
0811 02:57:58.424363 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.424413 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 02:57:58.424696 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.424393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:37941)
0811 02:57:58.424594 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.424639 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 02:57:58.425248 23364 heartbeater.cc:503] Master 127.8.203.125:33109 was elected leader, sending a full tablet report...
I20260811 02:57:58.425987 23546 tablet_bootstrap.cc:492] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: No bootstrap required, opened a new log
I20260811 02:57:58.426065 23546 ts_tablet_manager.cc:1397] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 02:57:58.426405 23546 raft_consensus.cc:348] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.426491 23546 raft_consensus.cc:374] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:57:58.426519 23546 raft_consensus.cc:729] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9d192581a374759a177fd9e50877f1a, State: Initialized, Role: FOLLOWER
I20260811 02:57:58.426630 23546 consensus_queue.cc:260] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.426920 23546 ts_tablet_manager.cc:1428] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:57:58.427400 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.427159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:47485)
0811 02:57:58.427273 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.427327 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 02:57:58.428176 23130 heartbeater.cc:503] Master 127.8.203.125:33109 was elected leader, sending a full tablet report...
I20260811 02:57:58.428259 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.428009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:55559)
0811 02:57:58.428148 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.428195 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 02:57:58.612322 23556 raft_consensus.cc:482] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:57:58.612489 23556 raft_consensus.cc:504] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.612910 23556 leader_election.cc:288] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265), 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583)
I20260811 02:57:58.613389 23272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "251176ddb9d845bea8e40d24d13be928" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" is_pre_election: true
I20260811 02:57:58.613392 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "251176ddb9d845bea8e40d24d13be928" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" is_pre_election: true
I20260811 02:57:58.613541 22783 raft_consensus.cc:2454] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 0.
I20260811 02:57:58.613545 23272 raft_consensus.cc:2454] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 0.
I20260811 02:57:58.613878 22937 leader_election.cc:302] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9d192581a374759a177fd9e50877f1a, bfd1d5297db24ff58c7db4a7df308b7d; no voters:
I20260811 02:57:58.614045 23556 raft_consensus.cc:2781] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:57:58.614114 23556 raft_consensus.cc:482] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:57:58.614158 23556 raft_consensus.cc:3037] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:58.615165 23556 raft_consensus.cc:504] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.615483 23556 leader_election.cc:288] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 election: Requested vote from peers bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265), 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583)
I20260811 02:57:58.615825 23272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "251176ddb9d845bea8e40d24d13be928" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b"
I20260811 02:57:58.615835 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "251176ddb9d845bea8e40d24d13be928" candidate_uuid: "b9d192581a374759a177fd9e50877f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd1d5297db24ff58c7db4a7df308b7d"
I20260811 02:57:58.615942 23272 raft_consensus.cc:3037] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:58.615969 22783 raft_consensus.cc:3037] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:57:58.616811 23272 raft_consensus.cc:2454] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 1.
I20260811 02:57:58.616882 22783 raft_consensus.cc:2454] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9d192581a374759a177fd9e50877f1a in term 1.
I20260811 02:57:58.617081 22932 leader_election.cc:302] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f9d4b5c0a994c33b72ed3a0931e2e6b, b9d192581a374759a177fd9e50877f1a; no voters:
I20260811 02:57:58.617262 23556 raft_consensus.cc:2781] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:57:58.617523 23556 raft_consensus.cc:686] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 1 LEADER]: Becoming Leader. State: Replica: b9d192581a374759a177fd9e50877f1a, State: Running, Role: LEADER
I20260811 02:57:58.617663 23556 consensus_queue.cc:237] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } }
I20260811 02:57:58.618182 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.617959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:43637)
0811 02:57:58.618061 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.618118 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 02:57:58.619514 22163 catalog_manager.cc:5495] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a reported cstate change: term changed from 0 to 1, leader changed from <none> to b9d192581a374759a177fd9e50877f1a (127.8.203.66). New cstate: current_term: 1 leader_uuid: "b9d192581a374759a177fd9e50877f1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9d192581a374759a177fd9e50877f1a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 35499 } health_report { overall_health: HEALTHY } } }
I20260811 02:57:58.649878 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.647028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.8.203.66:35499)
0811 02:57:58.647151 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:57:58.647156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:57:58.647169 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:57:58.647309 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:57:58.647312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:57:58.647362 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:57:58.647436 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:57:58.647658 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.648185 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.649015 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:57:58.649226 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:57:58.649319 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:57:58.649543 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:57:58.649611 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:57:58.649663 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:57:58.649727 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 02:57:58.649772 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:57:58.651813 22783 raft_consensus.cc:1261] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 1 FOLLOWER]: Refusing update from remote peer b9d192581a374759a177fd9e50877f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:58.651818 23272 raft_consensus.cc:1261] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Refusing update from remote peer b9d192581a374759a177fd9e50877f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:57:58.652230 23556 consensus_queue.cc:1035] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfd1d5297db24ff58c7db4a7df308b7d" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:58.652398 23561 consensus_queue.cc:1035] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9d4b5c0a994c33b72ed3a0931e2e6b" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:57:58.654383 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.654141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:60565)
0811 02:57:58.654242 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.654300 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 02:57:58.655876 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.654883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:39245)
0811 02:57:58.655746 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.655805 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":790}
I20260811 02:57:58.656756 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.656499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:32877)
0811 02:57:58.656638 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.656689 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 02:57:58.658391 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.658034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.67:38587)
0811 02:57:58.658268 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.658323 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260811 02:57:58.659061 23567 mvcc.cc:205] Tried to move back new op lower bound from 7317164354154037248 to 7317164354019332096. Current Snapshot: MvccSnapshot[applied={T|T < 7317164354154037248}]
I20260811 02:57:58.660305 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.660060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.66:44413)
0811 02:57:58.660193 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.660243 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 02:57:58.660458 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.659937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43163 (local address 127.8.203.65:60047)
0811 02:57:58.660319 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.660391 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:43163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313}
I20260811 02:57:58.662618 22166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:58.662751 22166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:57:58.666077 22166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6b74a505854495aba97aba8566cc46: Sending DeleteTablet for 3 replicas of tablet 251176ddb9d845bea8e40d24d13be928
I20260811 02:57:58.666755 23240 tablet_service.cc:1514] Processing DeleteTablet for tablet 251176ddb9d845bea8e40d24d13be928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:58 UTC) from {username='oryx', principal='oryx/127.8.203.125@KRBTEST.COM'} at 127.0.0.1:46486
I20260811 02:57:58.666939 23005 tablet_service.cc:1514] Processing DeleteTablet for tablet 251176ddb9d845bea8e40d24d13be928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:58 UTC) from {username='oryx', principal='oryx/127.8.203.125@KRBTEST.COM'} at 127.0.0.1:47138
I20260811 02:57:58.667086 23578 tablet_replica.cc:330] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: stopping tablet replica
I20260811 02:57:58.667222 23582 tablet_replica.cc:330] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: stopping tablet replica
I20260811 02:57:58.667227 22747 tablet_service.cc:1514] Processing DeleteTablet for tablet 251176ddb9d845bea8e40d24d13be928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:57:58 UTC) from {username='oryx', principal='oryx/127.8.203.125@KRBTEST.COM'} at 127.0.0.1:40698
I20260811 02:57:58.667356 23578 raft_consensus.cc:2229] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:58.667446 23582 raft_consensus.cc:2229] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:57:58.667511 23583 tablet_replica.cc:330] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d: stopping tablet replica
I20260811 02:57:58.667536 23578 raft_consensus.cc:2258] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:58.667649 23583 raft_consensus.cc:2229] T 251176ddb9d845bea8e40d24d13be928 P bfd1d5297db24ff58c7db4a7df308b7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:57:58.667672 23582 raft_consensus.cc:2258] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:57:58.667706 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 22445
I20260811 02:57:58.668726 23582 ts_tablet_manager.cc:1905] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:58.670733 23582 ts_tablet_manager.cc:1918] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:58.670784 23582 log.cc:1196] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/251176ddb9d845bea8e40d24d13be928
I20260811 02:57:58.670979 23582 ts_tablet_manager.cc:1939] T 251176ddb9d845bea8e40d24d13be928 P b9d192581a374759a177fd9e50877f1a: Deleting consensus metadata
I20260811 02:57:58.671591 22141 catalog_manager.cc:4841] TS b9d192581a374759a177fd9e50877f1a (127.8.203.66:35499): tablet 251176ddb9d845bea8e40d24d13be928 (table test-table [id=3cc4b34cca114648bfe996e02aaa0a51]) successfully deleted
I20260811 02:57:58.671645 23578 ts_tablet_manager.cc:1905] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:57:58.674316 23578 ts_tablet_manager.cc:1918] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:57:58.674486 23578 log.cc:1196] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/251176ddb9d845bea8e40d24d13be928
I20260811 02:57:58.674809 23578 ts_tablet_manager.cc:1939] T 251176ddb9d845bea8e40d24d13be928 P 3f9d4b5c0a994c33b72ed3a0931e2e6b: Deleting consensus metadata
I20260811 02:57:58.675457 22133 catalog_manager.cc:4841] TS 3f9d4b5c0a994c33b72ed3a0931e2e6b (127.8.203.67:42583): tablet 251176ddb9d845bea8e40d24d13be928 (table test-table [id=3cc4b34cca114648bfe996e02aaa0a51]) successfully deleted
I20260811 02:57:58.681341 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 22890
I20260811 02:57:58.686179 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:57:58.681412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:33265 (local address 127.0.0.1:40714)
0811 02:57:58.686051 (+ 4639us) negotiation.cc:107] Waiting for socket to connect
0811 02:57:58.686106 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4553}
W20260811 02:57:58.688095 22141 catalog_manager.cc:4584] TS bfd1d5297db24ff58c7db4a7df308b7d (127.8.203.65:33265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 251176ddb9d845bea8e40d24d13be928: Network error: Client connection negotiation failed: client connection to 127.8.203.65:33265: connect: Connection refused (error 111)
I20260811 02:57:58.702062 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 23137
I20260811 02:57:58.720556 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 21916
I20260811 02:57:58.734115 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 22103
2026-08-11T02:57:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7511 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:57:58 dist-test-slave-gjw7 krb5kdc[23586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:57:58 dist-test-slave-gjw7 krb5kdc[23586](info): set up 2 sockets
Aug 11 02:57:58 dist-test-slave-gjw7 krb5kdc[23586](info): commencing operation
krb5kdc: starting...
W20260811 02:57:59.941136 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:58:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab.
Entry for principal oryx/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab.
I20260811 02:58:00.060029 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:33651
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab
--principal=oryx/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:33651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:00.172183 23620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:00.172433 23620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:00.172472 23620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:00.176369 23620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:00.176417 23620 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:58:00.176433 23620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:00.176450 23620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:00.176465 23620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:00.176484 23620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:00.181571 23620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:33651
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.126.keytab
--principal=oryx/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:33651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:00.182487 23620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:00.183389 23620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:00.191545 23637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:00.191581 23620 server_base.cc:1034] running on GCE node
W20260811 02:58:00.191576 23638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:00.191732 23643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:00.192224 23620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:00.193233 23620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:00.194387 23620 hybrid_clock.cc:648] HybridClock initialized: now 1786417080194370 us; error 27 us; skew 500 ppm
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:00.196926 23620 init.cc:377] Logged in from keytab as oryx/127.8.203.126@KRBTEST.COM (short username oryx)
I20260811 02:58:00.197764 23620 webserver.cc:460] Webserver started at http://127.8.203.126:34369/ using document root <none> and password file <none>
I20260811 02:58:00.198292 23620 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:00.198356 23620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:00.198541 23620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:00.200475 23620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "092ba12eb7ae4cc99d84532bd04db4ec"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "8d49f28e25271304ab01f1d754c71ee3"
server_key_iv: "64311e9cebefc2b41bb0046924a88983"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.201043 23620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "092ba12eb7ae4cc99d84532bd04db4ec"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "8d49f28e25271304ab01f1d754c71ee3"
server_key_iv: "64311e9cebefc2b41bb0046924a88983"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.204578 23620 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:58:00.206796 23651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:00.207912 23620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:00.208022 23620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "092ba12eb7ae4cc99d84532bd04db4ec"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "8d49f28e25271304ab01f1d754c71ee3"
server_key_iv: "64311e9cebefc2b41bb0046924a88983"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.208108 23620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:00.222157 23620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:00.224606 23620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:00.224764 23620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:00.231969 23620 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:33651
I20260811 02:58:00.231984 23781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:33651 every 8 connection(s)
I20260811 02:58:00.232921 23620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:00.235544 23783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:00.236370 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 23620
I20260811 02:58:00.236488 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:58:00.236773 9005 external_mini_cluster.cc:1428] Setting key a763d8a40f0d392e812bdbfd7eed34c9
I20260811 02:58:00.241705 23783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Bootstrap starting.
I20260811 02:58:00.244287 23783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:00.245151 23783 log.cc:824] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:58:00.246984 23783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: No bootstrap required, opened a new log
I20260811 02:58:00.250247 23783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33651 } }
I20260811 02:58:00.250465 23783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:00.250524 23783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 092ba12eb7ae4cc99d84532bd04db4ec, State: Initialized, Role: FOLLOWER
I20260811 02:58:00.250737 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.238091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.8.203.126:33651)
0811 02:58:00.238478 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.238493 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.238529 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:58:00.239186 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.239205 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.239532 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.239796 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.240666 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.241656 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.242770 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.243216 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.243326 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.246040 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.246045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.246058 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.246083 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.248032 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.248510 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.248514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.248518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.248636 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.248905 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.248907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.248908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.249261 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:00.249464 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:00.249693 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 02:58:00.249951 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
I20260811 02:58:00.250959 23783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33651 } }
I20260811 02:58:00.251091 23783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:00.251155 23783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:00.251273 23783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:00.252269 23783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33651 } }
I20260811 02:58:00.252617 23783 leader_election.cc:302] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 092ba12eb7ae4cc99d84532bd04db4ec; no voters:
I20260811 02:58:00.252942 23783 leader_election.cc:288] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:00.253039 23789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:00.253228 23789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [term 1 LEADER]: Becoming Leader. State: Replica: 092ba12eb7ae4cc99d84532bd04db4ec, State: Running, Role: LEADER
I20260811 02:58:00.253633 23789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33651 } }
I20260811 02:58:00.254123 23783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:00.255352 23790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33651 } } }
I20260811 02:58:00.255338 23794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 092ba12eb7ae4cc99d84532bd04db4ec. Latest consensus state: current_term: 1 leader_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "092ba12eb7ae4cc99d84532bd04db4ec" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33651 } } }
I20260811 02:58:00.255476 23790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:00.255476 23794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:00.256500 23805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:00.259081 23805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:00.264482 23805 catalog_manager.cc:1347] Generated new cluster ID: 0d4d31eb1a5a4140b1a2068515997e10
I20260811 02:58:00.264539 23805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:00.280409 23805 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:58:00.281606 23805 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:00.291435 23805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Generated new TSK 0
I20260811 02:58:00.292213 23805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab.
Entry for principal oryx/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab.
I20260811 02:58:00.328037 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33651
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab
--principal=oryx/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:00.441684 23827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:00.441903 23827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:00.441943 23827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:00.445744 23827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:00.445785 23827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:00.445819 23827 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:58:00.445854 23827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:58:00.450678 23827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab
--principal=oryx/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:00.451558 23827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:00.452464 23827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:58:00.461773 23827 server_base.cc:1034] running on GCE node
W20260811 02:58:00.461776 23838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:00.461823 23843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:00.461927 23839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:00.462394 23827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:00.462978 23827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:00.464129 23827 hybrid_clock.cc:648] HybridClock initialized: now 1786417080464107 us; error 33 us; skew 500 ppm
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:00.466675 23827 init.cc:377] Logged in from keytab as oryx/127.8.203.65@KRBTEST.COM (short username oryx)
I20260811 02:58:00.467470 23827 webserver.cc:460] Webserver started at http://127.8.203.65:44721/ using document root <none> and password file <none>
I20260811 02:58:00.467995 23827 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:00.468062 23827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:00.468250 23827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:00.470288 23827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "707bc626bbd948e1b422f4c907be3cf8"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "7b24c6fc731d2745e5ff3107bb5435dc"
server_key_iv: "a2f72b0d075431cb8baa4674d3636656"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.470820 23827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "707bc626bbd948e1b422f4c907be3cf8"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "7b24c6fc731d2745e5ff3107bb5435dc"
server_key_iv: "a2f72b0d075431cb8baa4674d3636656"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.474383 23827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 02:58:00.476781 23854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:00.477970 23827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:58:00.478078 23827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "707bc626bbd948e1b422f4c907be3cf8"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "7b24c6fc731d2745e5ff3107bb5435dc"
server_key_iv: "a2f72b0d075431cb8baa4674d3636656"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.478171 23827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:00.506671 23827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:00.509413 23827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:00.509584 23827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:00.511108 23827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:00.511160 23827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:00.511214 23827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:00.511247 23827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:58:00.521450 23827 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:40063
I20260811 02:58:00.521646 24086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:40063 every 8 connection(s)
I20260811 02:58:00.522526 23827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:00.524416 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.511775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:51559 (local address 127.8.203.126:33651)
0811 02:58:00.511896 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.511901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.512969 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 02:58:00.513923 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.513927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.513986 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.514066 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.515589 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.516382 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.517897 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.518143 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.518233 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.520615 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.520619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.520622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.520649 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.522314 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.522827 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.522831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.522833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.522943 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.523315 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.523319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.523321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.523516 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:00.523605 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:00.524109 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 02:58:00.524234 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 02:58:00.524639 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 23827
I20260811 02:58:00.524752 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:00.525023 9005 external_mini_cluster.cc:1428] Setting key 510eecd659370d6fcfd51b2d917e1ff6
I20260811 02:58:00.525199 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.512335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33651 (local address 127.8.203.65:51559)
0811 02:58:00.512820 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:00.512864 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:00.513712 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:00.514358 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:00.514368 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:00.514811 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:00.515431 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.515454 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.516495 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.516499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.517774 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.517785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.518299 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.518303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.518474 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.518993 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:00.519009 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:00.520444 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:00.522423 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.522429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.522443 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.522729 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.523056 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.523060 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.523062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.523232 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.523624 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:00.523630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:00.523876 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 02:58:00.524037 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 02:58:00.524334 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":148,"threads_started":1}
I20260811 02:58:00.526538 24087 heartbeater.cc:347] Connected to a master server at 127.8.203.126:33651
I20260811 02:58:00.526669 24087 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:00.527204 24087 heartbeater.cc:511] Master 127.8.203.126:33651 requested a full tablet report, sending...
I20260811 02:58:00.528851 23688 ts_manager.cc:194] Registered new tserver with Master: 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063)
I20260811 02:58:00.530413 23688 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:51559
I20260811 02:58:00.535557 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.528592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33651 (local address 127.8.203.65:49617)
0811 02:58:00.528885 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:00.528896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:00.528984 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:00.529199 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:00.529202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:00.529407 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:00.529498 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.529503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.530304 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.530308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.531111 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.531120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.531639 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.531641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.531710 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.532138 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:00.532148 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:00.532432 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:00.534215 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.534218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.534219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.534403 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.534661 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.534663 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.534664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.534728 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.535088 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:00.535090 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:00.535157 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 02:58:00.535248 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 02:58:00.535378 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":88,"threads_started":1}
I20260811 02:58:00.535586 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.528653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:49617 (local address 127.8.203.126:33651)
0811 02:58:00.528757 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.528761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.529033 (+ 272us) server_negotiation.cc:407] Connection header received
0811 02:58:00.529072 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.529074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.529108 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.529170 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.529602 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.530191 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.531218 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.531529 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.531625 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.532564 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.532568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.532571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.532595 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.534103 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.534491 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.534495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.534497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.534583 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.534804 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.534807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.534809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.535008 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:00.535109 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:00.535316 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 02:58:00.535435 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":44}
WARNING: no policy specified for oryx/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab.
Entry for principal oryx/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab.
I20260811 02:58:00.554320 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33651
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab
--principal=oryx/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:00.661948 24101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:00.662163 24101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:00.662201 24101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:00.666007 24101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:00.666049 24101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:00.666083 24101 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:58:00.666118 24101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:58:00.671017 24101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab
--principal=oryx/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:00.671911 24101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:00.672820 24101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:00.681949 24111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:00.681962 24110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:00.682092 24101 server_base.cc:1034] running on GCE node
W20260811 02:58:00.682231 24115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:00.682591 24101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:00.683161 24101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:00.684317 24101 hybrid_clock.cc:648] HybridClock initialized: now 1786417080684301 us; error 26 us; skew 500 ppm
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:00.686848 24101 init.cc:377] Logged in from keytab as oryx/127.8.203.66@KRBTEST.COM (short username oryx)
I20260811 02:58:00.687647 24101 webserver.cc:460] Webserver started at http://127.8.203.66:44611/ using document root <none> and password file <none>
I20260811 02:58:00.688134 24101 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:00.688189 24101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:00.688352 24101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:00.690296 24101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "e1f4bea25f90448ea95ff41a06a0e267"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "9abbd85298b6184f240fdf04038d6eaf"
server_key_iv: "b3c4e4e7e461e7fc0eb35e7defde605a"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.690788 24101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "e1f4bea25f90448ea95ff41a06a0e267"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "9abbd85298b6184f240fdf04038d6eaf"
server_key_iv: "b3c4e4e7e461e7fc0eb35e7defde605a"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.694134 24101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:58:00.696322 24128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:00.697495 24101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:58:00.697589 24101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "e1f4bea25f90448ea95ff41a06a0e267"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "9abbd85298b6184f240fdf04038d6eaf"
server_key_iv: "b3c4e4e7e461e7fc0eb35e7defde605a"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.697680 24101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:00.731019 24101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:00.733762 24101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:00.733932 24101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:00.735366 24101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:00.735425 24101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:00.735481 24101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:00.735509 24101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:58:00.745687 24101 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:38795
I20260811 02:58:00.745827 24343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:38795 every 8 connection(s)
I20260811 02:58:00.746905 24101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:58:00.748668 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.736204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:55497 (local address 127.8.203.126:33651)
0811 02:58:00.736317 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.736321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.737423 (+ 1102us) server_negotiation.cc:407] Connection header received
0811 02:58:00.738474 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.738478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.738540 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.738628 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.740168 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.740769 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.742458 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.742733 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.742829 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.745210 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.745214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.745217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.745245 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.746785 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.747284 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.747288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.747290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.747389 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.747763 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.747767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.747768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.747961 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:00.748044 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:00.748456 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 02:58:00.748549 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:58:00.749475 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.736730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33651 (local address 127.8.203.66:55497)
0811 02:58:00.737240 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:00.737322 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:00.738196 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:00.738850 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:00.738861 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:00.739326 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:00.739986 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.740008 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.740893 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.740897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.742314 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.742326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.742892 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.742897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.743047 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.743600 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:00.743617 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:00.745036 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:00.746894 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.746899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.746910 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.747186 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.747502 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.747505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.747507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.747677 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.748045 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:00.748049 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:00.748257 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 02:58:00.748406 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 02:58:00.748638 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":148,"threads_started":1}
I20260811 02:58:00.750382 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 24101
I20260811 02:58:00.750496 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:58:00.750793 24344 heartbeater.cc:347] Connected to a master server at 127.8.203.126:33651
I20260811 02:58:00.750803 9005 external_mini_cluster.cc:1428] Setting key b091f278b29c32650e25f52e29a74485
I20260811 02:58:00.750938 24344 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:00.752113 24344 heartbeater.cc:511] Master 127.8.203.126:33651 requested a full tablet report, sending...
I20260811 02:58:00.753544 23685 ts_manager.cc:194] Registered new tserver with Master: e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795)
I20260811 02:58:00.754243 23685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:55497
I20260811 02:58:00.761073 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.752842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33651 (local address 127.8.203.66:53043)
0811 02:58:00.753094 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:00.753107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:00.753225 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:00.754761 (+ 1536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:00.754764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:00.754949 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:00.755060 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.755066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.756107 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.756110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.756702 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.756711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.757238 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.757241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.757356 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.757767 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:00.757777 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:00.758057 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:00.759755 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.759758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.759759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.759943 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.760206 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.760208 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.760209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.760273 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.760705 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:00.760707 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:00.760779 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 02:58:00.760855 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 02:58:00.760951 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
I20260811 02:58:00.761188 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.752918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:53043 (local address 127.8.203.126:33651)
0811 02:58:00.753024 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.753028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.753161 (+ 133us) server_negotiation.cc:407] Connection header received
0811 02:58:00.754522 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.754524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.754560 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.754652 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.755192 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.755990 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.756818 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.757135 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.757232 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.758189 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.758192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.758195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.758219 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.759657 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.760032 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.760035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.760037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.760125 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.760348 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.760351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.760353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.760615 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:00.760718 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:00.760938 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 02:58:00.761046 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
WARNING: no policy specified for oryx/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab.
Entry for principal oryx/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab.
I20260811 02:58:00.779251 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33651
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab
--principal=oryx/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:00.886919 24375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:00.887144 24375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:00.887184 24375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:00.890970 24375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:00.891014 24375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:00.891050 24375 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:58:00.891085 24375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:58:00.895910 24375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab
--principal=oryx/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.203.126:33651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:00.896827 24375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:00.897745 24375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:00.906863 24390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:00.906883 24386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:00.906869 24384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:00.906906 24375 server_base.cc:1034] running on GCE node
I20260811 02:58:00.907476 24375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:00.908082 24375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:00.909247 24375 hybrid_clock.cc:648] HybridClock initialized: now 1786417080909230 us; error 27 us; skew 500 ppm
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:00.911865 24375 init.cc:377] Logged in from keytab as oryx/127.8.203.67@KRBTEST.COM (short username oryx)
I20260811 02:58:00.912758 24375 webserver.cc:460] Webserver started at http://127.8.203.67:42173/ using document root <none> and password file <none>
I20260811 02:58:00.913343 24375 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:00.913420 24375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:00.913640 24375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:00.915674 24375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "c1b3230fa47e4bb4b004cc3d04e10b94"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "6ff0eeea145b4419e5c8e5e695c6cafa"
server_key_iv: "b743202b39e7c35b78a0219f90a57454"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.916220 24375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "c1b3230fa47e4bb4b004cc3d04e10b94"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "6ff0eeea145b4419e5c8e5e695c6cafa"
server_key_iv: "b743202b39e7c35b78a0219f90a57454"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.919786 24375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:58:00.922083 24398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:00.923213 24375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:58:00.923327 24375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "c1b3230fa47e4bb4b004cc3d04e10b94"
format_stamp: "Formatted at 2026-08-11 02:58:00 on dist-test-slave-gjw7"
server_key: "6ff0eeea145b4419e5c8e5e695c6cafa"
server_key_iv: "b743202b39e7c35b78a0219f90a57454"
server_key_version: "encryptionkey@0"
I20260811 02:58:00.923429 24375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:00.943246 24375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:00.946069 24375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:00.946240 24375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:00.947703 24375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:00.947753 24375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:00.947808 24375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:00.947835 24375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:58:00.958074 24375 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:42251
I20260811 02:58:00.958153 24604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:42251 every 8 connection(s)
I20260811 02:58:00.959048 24375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:58:00.961053 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.948507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:43087 (local address 127.8.203.126:33651)
0811 02:58:00.948627 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.948631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.949679 (+ 1048us) server_negotiation.cc:407] Connection header received
0811 02:58:00.950733 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.950737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.950799 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.950857 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.952553 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.953417 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.954824 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.955067 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.955160 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.957769 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.957773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.957776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.957805 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.959239 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.959787 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.959791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.959793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.959883 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.960182 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.960184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.960185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.960355 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:00.960427 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:00.960815 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 02:58:00.960933 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:58:00.961833 24416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.949044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33651 (local address 127.8.203.67:43087)
0811 02:58:00.949530 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:00.949572 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:00.950478 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:00.951320 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:00.951329 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:00.951731 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:00.952392 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.952413 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.953565 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.953569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.954704 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.954716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.955201 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.955206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.955386 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.955933 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:00.955950 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:00.957609 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:00.959350 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.959357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.959382 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.959692 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.959975 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.959978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.959980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.960109 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.960429 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:00.960433 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:00.960639 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 02:58:00.960781 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 02:58:00.961006 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":160,"threads_started":1}
I20260811 02:58:00.963014 24606 heartbeater.cc:347] Connected to a master server at 127.8.203.126:33651
I20260811 02:58:00.963150 24606 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:00.963637 24606 heartbeater.cc:511] Master 127.8.203.126:33651 requested a full tablet report, sending...
I20260811 02:58:00.964648 23688 ts_manager.cc:194] Registered new tserver with Master: c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251)
I20260811 02:58:00.964721 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 24375
I20260811 02:58:00.964848 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:58:00.965174 9005 external_mini_cluster.cc:1428] Setting key 45dac4c03e716e33cfe2cfccbfece0d0
I20260811 02:58:00.965365 23688 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:43087
I20260811 02:58:00.967658 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:58:00.971813 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.965522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:33355 (local address 127.8.203.126:33651)
0811 02:58:00.965634 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.965638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.965650 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:58:00.965683 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.965686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.965726 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.965791 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.966331 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.966799 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.967854 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.968058 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.968142 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.969121 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.969124 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.969125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.969146 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.970387 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.970762 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.970765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.970766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.970845 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:00.971123 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:00.971126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:00.971127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:00.971290 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:00.971366 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:00.971624 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 02:58:00.971709 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:58:00.971881 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.964957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33651 (local address 127.8.203.67:33355)
0811 02:58:00.965238 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:00.965252 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:00.965355 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:00.965899 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:00.965902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:00.966101 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:00.966218 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.966226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.966928 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.966931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.967694 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:00.967705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.968162 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.968165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:00.968265 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.968684 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:00.968695 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:00.968986 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:00.970491 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.970494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.970496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.970673 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.970929 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:00.970932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:00.970934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:00.971023 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:00.971390 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:00.971392 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:00.971482 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 02:58:00.971583 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 02:58:00.971705 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:58:00.992461 23787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:00.986764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.8.203.126:33651)
0811 02:58:00.986901 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.986905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.986917 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:58:00.986957 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.986959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.986997 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.987066 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.987792 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.988308 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.989149 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.989412 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.989503 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.992336 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51082: BlockingRecv error: recv got EOF from 127.0.0.1:51082 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 02:58:00 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:58:01.001029 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:00.993912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.8.203.126:33651)
0811 02:58:00.994028 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:00.994032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:00.994165 (+ 133us) server_negotiation.cc:407] Connection header received
0811 02:58:00.994266 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:00.994269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:00.994310 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:00.994375 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:00.994936 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.995417 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.996182 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:00.996390 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:00.996474 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:00.998609 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:00.998612 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:00.998615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:00.998637 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:00.999688 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:01.000035 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:01.000037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:01.000038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:01.000101 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:01.000348 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:01.000350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:01.000351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:01.000524 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:01.000623 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.000798 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.000886 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:58:01.003608 23688 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:58:01.006613 23688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:58:01.020862 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.015169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.8.203.67:42251)
0811 02:58:01.015517 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.015529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.015727 (+ 198us) server_negotiation.cc:407] Connection header received
0811 02:58:01.015954 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.015960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.016176 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.016309 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:01.016826 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.018059 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.019389 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.020150 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.020262 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.020431 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.020605 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.020706 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":123,"threads_started":1}
I20260811 02:58:01.020949 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.014863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.8.203.65:40063)
0811 02:58:01.015637 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.015653 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.015678 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:58:01.016313 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.016318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.016478 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.016616 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:01.016873 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.018075 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.019566 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.020364 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.020509 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.020666 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.020717 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.020803 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":76,"threads_started":1}
I20260811 02:58:01.020956 24636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.014705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40063 (local address 127.0.0.1:50200)
0811 02:58:01.015531 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.015579 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.015703 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.016609 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.016612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.016630 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:01.016733 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.016739 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.018171 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.018174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.019438 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.019448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.020510 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.020514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.020605 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.020627 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.020690 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.020825 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":73,"thread_start_us":56,"threads_started":1}
I20260811 02:58:01.021234 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.015375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38795 (local address 127.0.0.1:56278)
0811 02:58:01.015847 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.015860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.015964 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.016981 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.016983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.016998 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:01.017064 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.017068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.018049 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.018053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.019603 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.019611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.020576 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.020580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.020659 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.020682 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.020742 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.021118 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":543,"thread_start_us":84,"threads_started":1}
I20260811 02:58:01.021236 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.015900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56278 (local address 127.8.203.66:38795)
0811 02:58:01.016419 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.016429 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.016449 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:58:01.016520 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.016524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.016698 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.016850 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:01.017174 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.017936 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.019710 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.020247 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.020343 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.020515 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.020821 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.020908 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":124,"threads_started":1}
I20260811 02:58:01.021979 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.014818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42251 (local address 127.0.0.1:52148)
0811 02:58:01.015647 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.015657 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.015790 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.016516 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.016522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.016545 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:01.016661 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.016667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.018211 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.018214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.019267 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.019277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.020283 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.020287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.020392 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.020429 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.021586 (+ 1157us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.021661 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260811 02:58:01.023969 23949 tablet_service.cc:1467] Processing CreateTablet for tablet 11ea2485692440ceace43db8830d46a3 (DEFAULT_TABLE table=test-table [id=5546cc59599749bfbb5ac7e224100854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:01.023979 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 11ea2485692440ceace43db8830d46a3 (DEFAULT_TABLE table=test-table [id=5546cc59599749bfbb5ac7e224100854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:01.024358 24481 tablet_service.cc:1467] Processing CreateTablet for tablet 11ea2485692440ceace43db8830d46a3 (DEFAULT_TABLE table=test-table [id=5546cc59599749bfbb5ac7e224100854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:01.024977 23949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ea2485692440ceace43db8830d46a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.024991 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ea2485692440ceace43db8830d46a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.025250 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11ea2485692440ceace43db8830d46a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.032390 24654 tablet_bootstrap.cc:492] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Bootstrap starting.
I20260811 02:58:01.034368 24654 tablet_bootstrap.cc:654] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.034734 24655 tablet_bootstrap.cc:492] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Bootstrap starting.
I20260811 02:58:01.035138 24654 log.cc:824] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:01.035234 24653 tablet_bootstrap.cc:492] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Bootstrap starting.
I20260811 02:58:01.036803 24655 tablet_bootstrap.cc:654] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.037137 24654 tablet_bootstrap.cc:492] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: No bootstrap required, opened a new log
I20260811 02:58:01.037341 24654 ts_tablet_manager.cc:1397] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:01.037645 24653 tablet_bootstrap.cc:654] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.037644 24655 log.cc:824] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:01.038390 24653 log.cc:824] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:01.039474 24655 tablet_bootstrap.cc:492] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: No bootstrap required, opened a new log
I20260811 02:58:01.039652 24655 ts_tablet_manager.cc:1397] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:58:01.040210 24653 tablet_bootstrap.cc:492] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: No bootstrap required, opened a new log
I20260811 02:58:01.040392 24653 ts_tablet_manager.cc:1397] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:01.040614 24654 raft_consensus.cc:348] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.040824 24654 raft_consensus.cc:374] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.040886 24654 raft_consensus.cc:729] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f4bea25f90448ea95ff41a06a0e267, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.041308 24654 consensus_queue.cc:260] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.041941 24344 heartbeater.cc:503] Master 127.8.203.126:33651 was elected leader, sending a full tablet report...
I20260811 02:58:01.042219 24654 ts_tablet_manager.cc:1428] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:58:01.043304 24655 raft_consensus.cc:348] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.043550 24655 raft_consensus.cc:374] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.043603 24655 raft_consensus.cc:729] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.044023 24655 consensus_queue.cc:260] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.044653 24606 heartbeater.cc:503] Master 127.8.203.126:33651 was elected leader, sending a full tablet report...
I20260811 02:58:01.044951 24655 ts_tablet_manager.cc:1428] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:58:01.045006 24653 raft_consensus.cc:348] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.045239 24653 raft_consensus.cc:374] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.045317 24653 raft_consensus.cc:729] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707bc626bbd948e1b422f4c907be3cf8, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.045743 24653 consensus_queue.cc:260] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.046401 24087 heartbeater.cc:503] Master 127.8.203.126:33651 was elected leader, sending a full tablet report...
I20260811 02:58:01.046631 24653 ts_tablet_manager.cc:1428] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 02:58:01.210002 24607 tablet.cc:2369] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:58:01.248096 24346 tablet.cc:2369] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:58:01.273865 24088 tablet.cc:2369] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:58:01.279968 24667 raft_consensus.cc:482] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:01.280164 24667 raft_consensus.cc:504] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.281433 24667 leader_election.cc:288] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251)
I20260811 02:58:01.286526 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.281885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:60269 (local address 127.8.203.67:42251)
0811 02:58:01.282050 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.282054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.282073 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:58:01.282259 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.282263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.282313 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.282394 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:01.282761 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.283577 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.285325 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.285929 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.286049 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.286101 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.286327 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.286398 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 02:58:01.286526 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.281803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:47315 (local address 127.8.203.65:40063)
0811 02:58:01.282064 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.282068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.282159 (+ 91us) server_negotiation.cc:407] Connection header received
0811 02:58:01.282374 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.282377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.282428 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.282498 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:01.282760 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.283559 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.285320 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.285927 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.286049 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.286101 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.286326 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.286397 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260811 02:58:01.286583 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.281749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40063 (local address 127.8.203.66:47315)
0811 02:58:01.282100 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.282111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.282183 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.282518 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.282520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.282534 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:01.282657 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.282663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.283690 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.283694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.285165 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.285177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.286047 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.286051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.286197 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.286227 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.286301 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.286403 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":179,"thread_start_us":76,"threads_started":1}
I20260811 02:58:01.286588 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.281726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42251 (local address 127.8.203.66:60269)
0811 02:58:01.282000 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.282015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.282126 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.282442 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.282445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.282465 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:01.282643 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.282652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.283690 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.283694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.285165 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.285177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.286047 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.286051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.286194 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.286225 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.286301 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.286411 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":161,"thread_start_us":99,"threads_started":1}
I20260811 02:58:01.287288 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ea2485692440ceace43db8830d46a3" candidate_uuid: "e1f4bea25f90448ea95ff41a06a0e267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" is_pre_election: true
I20260811 02:58:01.287288 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ea2485692440ceace43db8830d46a3" candidate_uuid: "e1f4bea25f90448ea95ff41a06a0e267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8" is_pre_election: true
I20260811 02:58:01.287606 24518 raft_consensus.cc:2454] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f4bea25f90448ea95ff41a06a0e267 in term 0.
I20260811 02:58:01.287606 23989 raft_consensus.cc:2454] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1f4bea25f90448ea95ff41a06a0e267 in term 0.
I20260811 02:58:01.288115 24134 leader_election.cc:302] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, c1b3230fa47e4bb4b004cc3d04e10b94, e1f4bea25f90448ea95ff41a06a0e267; no voters:
I20260811 02:58:01.288384 24667 raft_consensus.cc:2781] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:01.288471 24667 raft_consensus.cc:482] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:01.288514 24667 raft_consensus.cc:3037] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:01.289544 24667 raft_consensus.cc:504] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.289942 24667 leader_election.cc:288] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [CANDIDATE]: Term 1 election: Requested vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251)
I20260811 02:58:01.290274 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ea2485692440ceace43db8830d46a3" candidate_uuid: "e1f4bea25f90448ea95ff41a06a0e267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8"
I20260811 02:58:01.290318 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11ea2485692440ceace43db8830d46a3" candidate_uuid: "e1f4bea25f90448ea95ff41a06a0e267" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94"
I20260811 02:58:01.290385 23989 raft_consensus.cc:3037] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:01.290418 24518 raft_consensus.cc:3037] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:01.291359 23989 raft_consensus.cc:2454] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f4bea25f90448ea95ff41a06a0e267 in term 1.
I20260811 02:58:01.291654 24134 leader_election.cc:302] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, e1f4bea25f90448ea95ff41a06a0e267; no voters:
I20260811 02:58:01.291834 24667 raft_consensus.cc:2781] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:01.291848 24518 raft_consensus.cc:2454] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1f4bea25f90448ea95ff41a06a0e267 in term 1.
I20260811 02:58:01.291992 24667 raft_consensus.cc:686] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 1 LEADER]: Becoming Leader. State: Replica: e1f4bea25f90448ea95ff41a06a0e267, State: Running, Role: LEADER
I20260811 02:58:01.292322 24667 consensus_queue.cc:237] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } }
I20260811 02:58:01.295261 23688 catalog_manager.cc:5495] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66). New cstate: current_term: 1 leader_uuid: "e1f4bea25f90448ea95ff41a06a0e267" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:01.311494 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.308065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.8.203.66:38795)
0811 02:58:01.308192 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.308196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.308210 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:58:01.308333 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.308336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.308397 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.308480 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:58:01.308716 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.309201 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.310045 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.310257 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.310343 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.310607 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:58:01.311130 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:58:01.311258 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.311315 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.311366 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:58:01.316923 23989 raft_consensus.cc:1261] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Refusing update from remote peer e1f4bea25f90448ea95ff41a06a0e267: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:01.317003 24518 raft_consensus.cc:1261] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Refusing update from remote peer e1f4bea25f90448ea95ff41a06a0e267: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:01.317654 24667 consensus_queue.cc:1035] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [LEADER]: Connected to new peer: Peer: permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:01.317860 24675 consensus_queue.cc:1035] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:01.325850 24682 mvcc.cc:205] Tried to move back new op lower bound from 7317164365067730944 to 7317164364976009216. Current Snapshot: MvccSnapshot[applied={T|T < 7317164365067730944}]
I20260811 02:58:01.329035 24684 mvcc.cc:205] Tried to move back new op lower bound from 7317164365067730944 to 7317164364976009216. Current Snapshot: MvccSnapshot[applied={T|T < 7317164365067730944 or (T in {7317164365067730944})}]
I20260811 02:58:01.329248 24683 mvcc.cc:205] Tried to move back new op lower bound from 7317164365067730944 to 7317164364976009216. Current Snapshot: MvccSnapshot[applied={T|T < 7317164365067730944}]
I20260811 02:58:01.334086 23685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:01.334252 23685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:01.336633 23685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Sending DeleteTablet for 3 replicas of tablet 11ea2485692440ceace43db8830d46a3
I20260811 02:58:01.337415 23949 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ea2485692440ceace43db8830d46a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:01 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:50200
I20260811 02:58:01.337487 24481 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ea2485692440ceace43db8830d46a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:01 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52148
I20260811 02:58:01.337602 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 11ea2485692440ceace43db8830d46a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:01 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:56278
I20260811 02:58:01.337949 24700 tablet_replica.cc:330] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: stopping tablet replica
I20260811 02:58:01.338073 24703 tablet_replica.cc:330] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: stopping tablet replica
I20260811 02:58:01.338387 24703 raft_consensus.cc:2229] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:58:01.338380 24700 raft_consensus.cc:2229] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:01.338634 23685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:58:01.338728 24700 raft_consensus.cc:2258] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:01.338765 24703 raft_consensus.cc:2258] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 02:58:01.339039 23685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:58:01.340309 24700 ts_tablet_manager.cc:1905] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:01.340309 24703 ts_tablet_manager.cc:1905] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:01.343175 24700 ts_tablet_manager.cc:1918] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:01.343163 24703 ts_tablet_manager.cc:1918] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:01.343314 24700 log.cc:1196] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/11ea2485692440ceace43db8830d46a3
I20260811 02:58:01.343319 24703 log.cc:1196] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/11ea2485692440ceace43db8830d46a3
I20260811 02:58:01.343699 24700 ts_tablet_manager.cc:1939] T 11ea2485692440ceace43db8830d46a3 P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting consensus metadata
I20260811 02:58:01.344125 23949 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbf8ab92d61445582a07aacec22901d (DEFAULT_TABLE table=test-table [id=fd774c1b62634fe7b48a1a148693d738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:01.344215 24481 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbf8ab92d61445582a07aacec22901d (DEFAULT_TABLE table=test-table [id=fd774c1b62634fe7b48a1a148693d738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:01.344446 23949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbf8ab92d61445582a07aacec22901d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.344481 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbf8ab92d61445582a07aacec22901d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.343704 24703 ts_tablet_manager.cc:1939] T 11ea2485692440ceace43db8830d46a3 P e1f4bea25f90448ea95ff41a06a0e267: Deleting consensus metadata
I20260811 02:58:01.345638 23656 catalog_manager.cc:4841] TS c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251): tablet 11ea2485692440ceace43db8830d46a3 (table test-table [id=5546cc59599749bfbb5ac7e224100854]) successfully deleted
I20260811 02:58:01.345992 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbf8ab92d61445582a07aacec22901d (DEFAULT_TABLE table=test-table [id=fd774c1b62634fe7b48a1a148693d738]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:01.346237 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbf8ab92d61445582a07aacec22901d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.348086 24655 tablet_bootstrap.cc:492] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: Bootstrap starting.
I20260811 02:58:01.348421 24702 tablet_replica.cc:330] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: stopping tablet replica
I20260811 02:58:01.348512 24653 tablet_bootstrap.cc:492] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: Bootstrap starting.
I20260811 02:58:01.348856 24702 raft_consensus.cc:2229] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:01.349116 24655 tablet_bootstrap.cc:654] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.349349 24702 raft_consensus.cc:2258] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:01.349442 24653 tablet_bootstrap.cc:654] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.349819 24654 tablet_bootstrap.cc:492] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: Bootstrap starting.
I20260811 02:58:01.350574 24653 tablet_bootstrap.cc:492] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: No bootstrap required, opened a new log
I20260811 02:58:01.350597 23656 catalog_manager.cc:4841] TS e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795): tablet 11ea2485692440ceace43db8830d46a3 (table test-table [id=5546cc59599749bfbb5ac7e224100854]) successfully deleted
I20260811 02:58:01.350670 24653 ts_tablet_manager.cc:1397] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:01.350804 24654 tablet_bootstrap.cc:654] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.351112 24653 raft_consensus.cc:348] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.351224 24653 raft_consensus.cc:374] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.351260 24653 raft_consensus.cc:729] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707bc626bbd948e1b422f4c907be3cf8, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.351370 24653 consensus_queue.cc:260] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.351611 24653 ts_tablet_manager.cc:1428] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:58:01.352110 24702 ts_tablet_manager.cc:1905] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:01.354038 24654 tablet_bootstrap.cc:492] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: No bootstrap required, opened a new log
I20260811 02:58:01.354127 24654 ts_tablet_manager.cc:1397] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:58:01.354621 24654 raft_consensus.cc:348] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.354740 24654 raft_consensus.cc:374] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.354773 24654 raft_consensus.cc:729] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f4bea25f90448ea95ff41a06a0e267, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.354889 24654 consensus_queue.cc:260] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.355144 24654 ts_tablet_manager.cc:1428] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:58:01.355182 24702 ts_tablet_manager.cc:1918] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:01.355273 24702 log.cc:1196] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/11ea2485692440ceace43db8830d46a3
I20260811 02:58:01.355618 24702 ts_tablet_manager.cc:1939] T 11ea2485692440ceace43db8830d46a3 P 707bc626bbd948e1b422f4c907be3cf8: Deleting consensus metadata
I20260811 02:58:01.356734 23657 catalog_manager.cc:4841] TS 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063): tablet 11ea2485692440ceace43db8830d46a3 (table test-table [id=5546cc59599749bfbb5ac7e224100854]) successfully deleted
I20260811 02:58:01.357575 24655 tablet_bootstrap.cc:492] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: No bootstrap required, opened a new log
I20260811 02:58:01.357672 24655 ts_tablet_manager.cc:1397] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260811 02:58:01.358148 24655 raft_consensus.cc:348] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.358356 24655 raft_consensus.cc:374] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.358394 24655 raft_consensus.cc:729] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.358541 24655 consensus_queue.cc:260] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.358873 24655 ts_tablet_manager.cc:1428] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:58:01.710491 24669 raft_consensus.cc:482] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:01.710665 24669 raft_consensus.cc:504] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.711866 24669 leader_election.cc:288] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795)
I20260811 02:58:01.715945 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.712280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:42517 (local address 127.8.203.66:38795)
0811 02:58:01.712477 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.712482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.712638 (+ 156us) server_negotiation.cc:407] Connection header received
0811 02:58:01.712810 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.712813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.712862 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.712911 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:01.713253 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.713927 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.714962 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.715351 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.715461 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.715520 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.715730 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.715800 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 02:58:01.715945 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.712236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:41347 (local address 127.8.203.65:40063)
0811 02:58:01.712365 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.712380 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.712572 (+ 192us) server_negotiation.cc:407] Connection header received
0811 02:58:01.712741 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.712744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.712796 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.712873 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:01.713163 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.713850 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.714958 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.715356 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.715461 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.715520 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.715733 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.715810 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:58:01.715963 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.712122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40063 (local address 127.8.203.67:41347)
0811 02:58:01.712466 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.712481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.712603 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.712909 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.712912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.712928 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:01.713050 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.713057 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.713954 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.713957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.714831 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.714849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.715461 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.715465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.715618 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.715641 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.715706 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.715802 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":66,"thread_start_us":88,"threads_started":1}
I20260811 02:58:01.715962 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.712122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38795 (local address 127.8.203.67:42517)
0811 02:58:01.712547 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.712560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.712678 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.713043 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.713045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.713060 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:01.713140 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.713146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.714039 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.714042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.714838 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.714846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.715464 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.715468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.715608 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.715634 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.715698 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.715789 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":132,"thread_start_us":67,"threads_started":1}
I20260811 02:58:01.716497 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbf8ab92d61445582a07aacec22901d" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8" is_pre_election: true
I20260811 02:58:01.716648 23989 raft_consensus.cc:2454] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 0.
I20260811 02:58:01.716799 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbf8ab92d61445582a07aacec22901d" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267" is_pre_election: true
I20260811 02:58:01.717034 24263 raft_consensus.cc:2454] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 0.
I20260811 02:58:01.717013 24406 leader_election.cc:302] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, c1b3230fa47e4bb4b004cc3d04e10b94; no voters:
I20260811 02:58:01.717263 24669 raft_consensus.cc:2781] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:01.717351 24669 raft_consensus.cc:482] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:01.717401 24669 raft_consensus.cc:3037] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:01.718374 24669 raft_consensus.cc:504] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.718712 24669 leader_election.cc:288] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 election: Requested vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795)
I20260811 02:58:01.719048 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbf8ab92d61445582a07aacec22901d" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267"
I20260811 02:58:01.719048 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbf8ab92d61445582a07aacec22901d" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8"
I20260811 02:58:01.719161 23989 raft_consensus.cc:3037] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:01.719169 24263 raft_consensus.cc:3037] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:01.720055 23989 raft_consensus.cc:2454] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 1.
I20260811 02:58:01.720088 24263 raft_consensus.cc:2454] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 1.
I20260811 02:58:01.720312 24406 leader_election.cc:302] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, c1b3230fa47e4bb4b004cc3d04e10b94; no voters:
I20260811 02:58:01.720476 24669 raft_consensus.cc:2781] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:01.720639 24669 raft_consensus.cc:686] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 LEADER]: Becoming Leader. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Running, Role: LEADER
I20260811 02:58:01.720866 24669 consensus_queue.cc:237] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.723186 23685 catalog_manager.cc:5495] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67). New cstate: current_term: 1 leader_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } health_report { overall_health: HEALTHY } } }
Aug 11 02:58:01 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417080, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.126@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:58:01.830668 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.822785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33651 (local address 127.0.0.1:51102)
0811 02:58:01.823059 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:01.823072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:01.823166 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:01.823378 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:01.823380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:01.823778 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:01.823879 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.823885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.824560 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.824562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.825105 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:01.825111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.825488 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.825491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:01.825550 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.826022 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:01.826033 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:01.827460 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:01.829127 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:01.829138 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:01.829144 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:01.829475 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:01.829724 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:01.829727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:01.829729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:01.829807 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:01.830158 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:01.830165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:01.830272 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 02:58:01.830404 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 02:58:01.830536 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
I20260811 02:58:01.830725 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:01.822837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.8.203.126:33651)
0811 02:58:01.823083 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:01.823089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:01.823121 (+ 32us) server_negotiation.cc:407] Connection header received
0811 02:58:01.823233 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:01.823235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:01.823274 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:01.823355 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:01.823962 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.824489 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.825176 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:01.825407 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:01.825469 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:01.827578 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:01.827581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:01.827583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:01.827606 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:01.829003 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:01.829564 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:01.829568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:01.829570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:01.829648 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:01.829887 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:01.829891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:01.829892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:01.830062 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:01.830145 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:01.830459 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 02:58:01.830603 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260811 02:58:01.834092 23685 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:51102:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:58:01.834764 23685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:58:01.839268 23949 tablet_service.cc:1467] Processing CreateTablet for tablet 4809b1adfe4a4910bbc05a17a54fe3d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb216c16cfa044e694d6ba31b9bdcd5c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:58:01.839273 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 4809b1adfe4a4910bbc05a17a54fe3d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb216c16cfa044e694d6ba31b9bdcd5c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:58:01.839345 24481 tablet_service.cc:1467] Processing CreateTablet for tablet 4809b1adfe4a4910bbc05a17a54fe3d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb216c16cfa044e694d6ba31b9bdcd5c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:58:01.839666 23949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4809b1adfe4a4910bbc05a17a54fe3d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.839666 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4809b1adfe4a4910bbc05a17a54fe3d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.839691 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4809b1adfe4a4910bbc05a17a54fe3d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:01.842015 24655 tablet_bootstrap.cc:492] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: Bootstrap starting.
I20260811 02:58:01.842375 24654 tablet_bootstrap.cc:492] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: Bootstrap starting.
I20260811 02:58:01.842494 24653 tablet_bootstrap.cc:492] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: Bootstrap starting.
I20260811 02:58:01.842968 24655 tablet_bootstrap.cc:654] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.843040 24654 tablet_bootstrap.cc:654] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.843355 24653 tablet_bootstrap.cc:654] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:01.844229 24654 tablet_bootstrap.cc:492] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: No bootstrap required, opened a new log
I20260811 02:58:01.844323 24654 ts_tablet_manager.cc:1397] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:01.844545 24653 tablet_bootstrap.cc:492] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: No bootstrap required, opened a new log
I20260811 02:58:01.844619 24653 ts_tablet_manager.cc:1397] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:01.844643 24655 tablet_bootstrap.cc:492] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: No bootstrap required, opened a new log
I20260811 02:58:01.844729 24655 ts_tablet_manager.cc:1397] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:58:01.844722 24654 raft_consensus.cc:348] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.844843 24654 raft_consensus.cc:374] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.844887 24654 raft_consensus.cc:729] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f4bea25f90448ea95ff41a06a0e267, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.845011 24654 consensus_queue.cc:260] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.845038 24653 raft_consensus.cc:348] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.845137 24653 raft_consensus.cc:374] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.845165 24653 raft_consensus.cc:729] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707bc626bbd948e1b422f4c907be3cf8, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.845162 24655 raft_consensus.cc:348] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.845329 24654 ts_tablet_manager.cc:1428] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:58:01.845276 24653 consensus_queue.cc:260] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.845271 24655 raft_consensus.cc:374] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:01.845377 24655 raft_consensus.cc:729] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Initialized, Role: FOLLOWER
I20260811 02:58:01.845539 24653 ts_tablet_manager.cc:1428] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:58:01.845506 24655 consensus_queue.cc:260] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:01.845526 24694 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:01.845655 24694 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:58:01.845759 24655 ts_tablet_manager.cc:1428] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:58:01.845747 24671 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:01.845862 24671 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:58:01.845932 24669 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:01.846051 24669 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:58:02.216076 24669 consensus_queue.cc:1035] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.219815 24671 raft_consensus.cc:482] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:02.219923 24671 raft_consensus.cc:504] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.220105 24741 consensus_queue.cc:1035] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.221035 24671 leader_election.cc:288] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795), c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251)
I20260811 02:58:02.225800 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.221593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:37935 (local address 127.8.203.66:38795)
0811 02:58:02.222022 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.222026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.222040 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:58:02.222293 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.222296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.222339 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.222408 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:02.222687 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.223392 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.224865 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.225274 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.225405 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.225461 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.225610 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.225676 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":50,"threads_started":1}
I20260811 02:58:02.225814 24749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.221540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38795 (local address 127.8.203.65:37935)
0811 02:58:02.221799 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:02.221814 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:02.221915 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:02.222421 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:02.222424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:02.222454 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:02.222559 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.222565 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.223511 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.223515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.224762 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.224773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.225409 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.225413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.225502 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.225525 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:58:02.225589 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 02:58:02.225662 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":72,"threads_started":1}
I20260811 02:58:02.226179 24752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.221854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42251 (local address 127.8.203.65:33607)
0811 02:58:02.222200 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:02.222214 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:02.222300 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:02.222565 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:02.222568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:02.222588 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:02.222668 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.222674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.224068 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.224071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.225088 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.225096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.225761 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.225765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.225878 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.225905 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:58:02.225975 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:58:02.226045 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":94,"threads_started":1}
I20260811 02:58:02.226255 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.222130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:33607 (local address 127.8.203.67:42251)
0811 02:58:02.222339 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.222344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.222360 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:58:02.222407 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.222409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.222474 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.222559 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:02.223168 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.223942 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.225194 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.225654 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.225761 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.225811 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.226014 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.226094 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":57,"threads_started":1}
I20260811 02:58:02.226243 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4809b1adfe4a4910bbc05a17a54fe3d8" candidate_uuid: "707bc626bbd948e1b422f4c907be3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267" is_pre_election: true
I20260811 02:58:02.226382 24263 raft_consensus.cc:2454] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707bc626bbd948e1b422f4c907be3cf8 in term 0.
I20260811 02:58:02.226647 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4809b1adfe4a4910bbc05a17a54fe3d8" candidate_uuid: "707bc626bbd948e1b422f4c907be3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" is_pre_election: true
I20260811 02:58:02.226725 23861 leader_election.cc:302] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, e1f4bea25f90448ea95ff41a06a0e267; no voters:
I20260811 02:58:02.226780 24518 raft_consensus.cc:2454] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707bc626bbd948e1b422f4c907be3cf8 in term 0.
I20260811 02:58:02.226991 24744 raft_consensus.cc:2781] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:02.227054 24744 raft_consensus.cc:482] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:02.227094 24744 raft_consensus.cc:3037] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.228132 24744 raft_consensus.cc:504] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.228433 24744 leader_election.cc:288] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [CANDIDATE]: Term 1 election: Requested vote from peers e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795), c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251)
I20260811 02:58:02.228765 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4809b1adfe4a4910bbc05a17a54fe3d8" candidate_uuid: "707bc626bbd948e1b422f4c907be3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267"
I20260811 02:58:02.228816 24518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4809b1adfe4a4910bbc05a17a54fe3d8" candidate_uuid: "707bc626bbd948e1b422f4c907be3cf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94"
I20260811 02:58:02.228873 24263 raft_consensus.cc:3037] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.228904 24518 raft_consensus.cc:3037] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.229769 24518 raft_consensus.cc:2454] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 707bc626bbd948e1b422f4c907be3cf8 in term 1.
I20260811 02:58:02.229787 24263 raft_consensus.cc:2454] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 707bc626bbd948e1b422f4c907be3cf8 in term 1.
I20260811 02:58:02.230011 23861 leader_election.cc:302] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, c1b3230fa47e4bb4b004cc3d04e10b94; no voters:
I20260811 02:58:02.230181 24744 raft_consensus.cc:2781] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:02.230289 24744 raft_consensus.cc:686] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [term 1 LEADER]: Becoming Leader. State: Replica: 707bc626bbd948e1b422f4c907be3cf8, State: Running, Role: LEADER
I20260811 02:58:02.230505 24744 consensus_queue.cc:237] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.231626 24671 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: TxnStatusTablet state changed. Reason: New leader 707bc626bbd948e1b422f4c907be3cf8. Latest consensus state: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:02.231731 24671 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:58:02.232672 23686 catalog_manager.cc:5495] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65). New cstate: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } health_report { overall_health: UNKNOWN } } }
I20260811 02:58:02.237450 24758 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:58:02.237586 24758 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:58:02.300078 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.296265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40063 (local address 127.0.0.1:50216)
0811 02:58:02.296421 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:02.296437 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:02.296532 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:02.297029 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:02.297032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:02.297039 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:58:02.297144 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.297149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.297937 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.297941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.298539 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.298546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.298977 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.298981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.299067 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.299112 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.299771 (+ 659us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.299787 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 02:58:02.299849 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 02:58:02.299947 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 02:58:02.300088 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.296394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50216 (local address 127.8.203.65:40063)
0811 02:58:02.296742 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.296748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.296759 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:58:02.296829 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.296831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.296887 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.296989 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:58:02.297244 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.297828 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.298645 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.298878 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.298966 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.299248 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.299681 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.299781 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.299860 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.299914 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":146,"threads_started":1}
I20260811 02:58:02.303237 24518 raft_consensus.cc:1261] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Refusing update from remote peer 707bc626bbd948e1b422f4c907be3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:02.303267 24263 raft_consensus.cc:1261] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Refusing update from remote peer 707bc626bbd948e1b422f4c907be3cf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:02.303781 24671 consensus_queue.cc:1035] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.304216 24744 consensus_queue.cc:1035] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.305655 24741 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: TxnStatusTablet state changed. Reason: New leader 707bc626bbd948e1b422f4c907be3cf8. Latest consensus state: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:02.305655 24694 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: TxnStatusTablet state changed. Reason: New leader 707bc626bbd948e1b422f4c907be3cf8. Latest consensus state: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:02.305768 24694 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:58:02.305768 24741 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:58:02.308259 24671 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:02.308290 24744 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:02.308364 24671 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:58:02.308399 24744 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P 707bc626bbd948e1b422f4c907be3cf8: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:58:02.309034 24694 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:02.309128 24694 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P e1f4bea25f90448ea95ff41a06a0e267: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:58:02.310351 24741 tablet_replica.cc:439] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "707bc626bbd948e1b422f4c907be3cf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } } }
I20260811 02:58:02.310457 24741 tablet_replica.cc:442] T 4809b1adfe4a4910bbc05a17a54fe3d8 P c1b3230fa47e4bb4b004cc3d04e10b94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:58:02.319219 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.315255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.8.203.67:42251)
0811 02:58:02.315377 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.315381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.315550 (+ 169us) server_negotiation.cc:407] Connection header received
0811 02:58:02.315684 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.315687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.315751 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.315860 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:58:02.316068 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.316845 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.317703 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.318047 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.318146 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.318330 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.318881 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.318990 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.319025 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.319081 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:58:02.330000 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.326376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40063 (local address 127.8.203.67:56417)
0811 02:58:02.326703 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:02.326717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:02.326819 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:02.327053 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:02.327055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:02.327082 (+ 27us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:58:02.327179 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.327184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.328120 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.328122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.328720 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.328729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.329259 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.329262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.329366 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.329422 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.329774 (+ 352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.329777 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:58:02.329817 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 02:58:02.329879 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":104,"threads_started":1}
I20260811 02:58:02.330013 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.326459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56417 (local address 127.8.203.65:40063)
0811 02:58:02.326590 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.326594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.326773 (+ 179us) server_negotiation.cc:407] Connection header received
0811 02:58:02.326924 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.326926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.326973 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.327035 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:58:02.327307 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.328021 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.328831 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.329154 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.329263 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.329586 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.329676 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.329777 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.329833 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.329892 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:58:02.356091 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.352147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:37563 (local address 127.8.203.67:42251)
0811 02:58:02.352415 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.352420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.352436 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:58:02.352554 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.352557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.352607 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.352702 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:58:02.352885 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.353654 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.354884 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.355225 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.355403 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.355662 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.355755 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.355833 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.355894 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.355959 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260811 02:58:02.356123 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.352119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:42251 (local address 127.8.203.65:37563)
0811 02:58:02.352364 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:02.352383 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:02.352461 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:02.352687 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:02.352689 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:02.352694 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:58:02.352800 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.352803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.353941 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.353944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.354761 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:02.354772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.355365 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.355368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:02.355488 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.355539 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.355833 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.355837 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:58:02.355881 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 02:58:02.355956 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260811 02:58:02.378518 23688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:02.378713 23688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:02.380887 23688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Sending DeleteTablet for 3 replicas of tablet 4cbf8ab92d61445582a07aacec22901d
I20260811 02:58:02.381486 23949 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cbf8ab92d61445582a07aacec22901d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:50200
I20260811 02:58:02.381817 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cbf8ab92d61445582a07aacec22901d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:56278
I20260811 02:58:02.381896 24805 tablet_replica.cc:330] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: stopping tablet replica
I20260811 02:58:02.382016 24805 raft_consensus.cc:2229] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:02.382112 24805 raft_consensus.cc:2258] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:02.382445 24481 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cbf8ab92d61445582a07aacec22901d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52148
I20260811 02:58:02.383038 24808 tablet_replica.cc:330] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: stopping tablet replica
I20260811 02:58:02.383194 24808 raft_consensus.cc:2229] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:58:02.385974 24808 raft_consensus.cc:2258] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:02.386186 24806 tablet_replica.cc:330] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: stopping tablet replica
I20260811 02:58:02.386511 24805 ts_tablet_manager.cc:1905] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:02.386848 24808 ts_tablet_manager.cc:1905] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:02.387054 24806 raft_consensus.cc:2229] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:02.387215 24806 raft_consensus.cc:2258] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:02.387867 24806 ts_tablet_manager.cc:1905] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:02.389622 24805 ts_tablet_manager.cc:1918] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:58:02.389762 24805 log.cc:1196] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/4cbf8ab92d61445582a07aacec22901d
I20260811 02:58:02.390074 24805 ts_tablet_manager.cc:1939] T 4cbf8ab92d61445582a07aacec22901d P 707bc626bbd948e1b422f4c907be3cf8: Deleting consensus metadata
I20260811 02:58:02.390666 24808 ts_tablet_manager.cc:1918] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:58:02.390990 23657 catalog_manager.cc:4841] TS 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063): tablet 4cbf8ab92d61445582a07aacec22901d (table test-table [id=fd774c1b62634fe7b48a1a148693d738]) successfully deleted
I20260811 02:58:02.393369 24806 ts_tablet_manager.cc:1918] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:58:02.393455 24806 log.cc:1196] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/4cbf8ab92d61445582a07aacec22901d
I20260811 02:58:02.393702 24806 ts_tablet_manager.cc:1939] T 4cbf8ab92d61445582a07aacec22901d P e1f4bea25f90448ea95ff41a06a0e267: Deleting consensus metadata
I20260811 02:58:02.394279 24808 log.cc:1196] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/4cbf8ab92d61445582a07aacec22901d
I20260811 02:58:02.394393 23656 catalog_manager.cc:4841] TS e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795): tablet 4cbf8ab92d61445582a07aacec22901d (table test-table [id=fd774c1b62634fe7b48a1a148693d738]) successfully deleted
I20260811 02:58:02.394605 24808 ts_tablet_manager.cc:1939] T 4cbf8ab92d61445582a07aacec22901d P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting consensus metadata
I20260811 02:58:02.395260 23656 catalog_manager.cc:4841] TS c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251): tablet 4cbf8ab92d61445582a07aacec22901d (table test-table [id=fd774c1b62634fe7b48a1a148693d738]) successfully deleted
Aug 11 02:58:02 dist-test-slave-gjw7 krb5kdc[23586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:58:02 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:02 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:58:02.419842 24835 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:02.412902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.8.203.126:33651)
0811 02:58:02.413151 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.413154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.413172 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:58:02.413411 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.413413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.413461 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.413538 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:02.414388 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.414914 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.416088 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.416296 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.416361 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.419690 (+ 3329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51116: BlockingRecv error: recv got EOF from 127.0.0.1:51116 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
Aug 11 02:58:02 dist-test-slave-gjw7 krb5kdc[23586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM
I20260811 02:58:02.429214 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.421212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.8.203.126:33651)
0811 02:58:02.421392 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.421396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.421468 (+ 72us) server_negotiation.cc:407] Connection header received
0811 02:58:02.421565 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.421567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.421610 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.421696 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:02.422169 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.422965 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.423664 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.424006 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.424107 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.426143 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:02.426148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:02.426150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:02.426177 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:02.427755 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:02.428120 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:02.428124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:02.428125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:02.428216 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:02.428426 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:02.428429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:02.428430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:02.428633 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:02.428777 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.428931 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.429050 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:58:02.431721 23688 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:58:02.432195 23688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:58:02.436956 23949 tablet_service.cc:1467] Processing CreateTablet for tablet 04e2d52d46ab4762b513faecd53a9b5f (DEFAULT_TABLE table=test-table [id=c5c4e6973a31430fa60103792df6a46b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:02.437004 24227 tablet_service.cc:1467] Processing CreateTablet for tablet 04e2d52d46ab4762b513faecd53a9b5f (DEFAULT_TABLE table=test-table [id=c5c4e6973a31430fa60103792df6a46b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:02.437270 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e2d52d46ab4762b513faecd53a9b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:02.437268 23949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e2d52d46ab4762b513faecd53a9b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:02.439221 24854 tablet_bootstrap.cc:492] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: Bootstrap starting.
I20260811 02:58:02.439793 24856 tablet_bootstrap.cc:492] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: Bootstrap starting.
I20260811 02:58:02.440115 24481 tablet_service.cc:1467] Processing CreateTablet for tablet 04e2d52d46ab4762b513faecd53a9b5f (DEFAULT_TABLE table=test-table [id=c5c4e6973a31430fa60103792df6a46b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:02.440186 24854 tablet_bootstrap.cc:654] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:02.440380 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04e2d52d46ab4762b513faecd53a9b5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:02.440560 24856 tablet_bootstrap.cc:654] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:02.441666 24854 tablet_bootstrap.cc:492] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: No bootstrap required, opened a new log
I20260811 02:58:02.441666 24856 tablet_bootstrap.cc:492] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: No bootstrap required, opened a new log
I20260811 02:58:02.441741 24856 ts_tablet_manager.cc:1397] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:02.441741 24854 ts_tablet_manager.cc:1397] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:58:02.442122 24854 raft_consensus.cc:348] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.442122 24856 raft_consensus.cc:348] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.442225 24854 raft_consensus.cc:374] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:02.442224 24856 raft_consensus.cc:374] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:02.442260 24854 raft_consensus.cc:729] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f4bea25f90448ea95ff41a06a0e267, State: Initialized, Role: FOLLOWER
I20260811 02:58:02.442260 24856 raft_consensus.cc:729] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707bc626bbd948e1b422f4c907be3cf8, State: Initialized, Role: FOLLOWER
I20260811 02:58:02.442378 24856 consensus_queue.cc:260] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.442410 24854 consensus_queue.cc:260] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.442633 24856 ts_tablet_manager.cc:1428] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:58:02.443063 24854 ts_tablet_manager.cc:1428] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:58:02.443877 24859 tablet_bootstrap.cc:492] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: Bootstrap starting.
I20260811 02:58:02.444796 24859 tablet_bootstrap.cc:654] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:02.446070 24859 tablet_bootstrap.cc:492] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: No bootstrap required, opened a new log
I20260811 02:58:02.446159 24859 ts_tablet_manager.cc:1397] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 02:58:02.446573 24859 raft_consensus.cc:348] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.446712 24859 raft_consensus.cc:374] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:02.446753 24859 raft_consensus.cc:729] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Initialized, Role: FOLLOWER
I20260811 02:58:02.446863 24859 consensus_queue.cc:260] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.447129 24859 ts_tablet_manager.cc:1428] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:58:02.753435 24713 raft_consensus.cc:482] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:02.753589 24713 raft_consensus.cc:504] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.754015 24713 leader_election.cc:288] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795)
I20260811 02:58:02.754441 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e2d52d46ab4762b513faecd53a9b5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267" is_pre_election: true
I20260811 02:58:02.754482 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e2d52d46ab4762b513faecd53a9b5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8" is_pre_election: true
I20260811 02:58:02.754602 24263 raft_consensus.cc:2454] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 0.
I20260811 02:58:02.754601 23989 raft_consensus.cc:2454] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 0.
I20260811 02:58:02.754920 24403 leader_election.cc:302] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b3230fa47e4bb4b004cc3d04e10b94, e1f4bea25f90448ea95ff41a06a0e267; no voters:
I20260811 02:58:02.755117 24713 raft_consensus.cc:2781] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:02.755183 24713 raft_consensus.cc:482] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:02.755223 24713 raft_consensus.cc:3037] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.756521 24713 raft_consensus.cc:504] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.756852 24713 leader_election.cc:288] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 election: Requested vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795)
I20260811 02:58:02.757181 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e2d52d46ab4762b513faecd53a9b5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267"
I20260811 02:58:02.757179 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04e2d52d46ab4762b513faecd53a9b5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8"
I20260811 02:58:02.757279 24263 raft_consensus.cc:3037] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.757309 23989 raft_consensus.cc:3037] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.758164 24263 raft_consensus.cc:2454] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 1.
I20260811 02:58:02.758364 23989 raft_consensus.cc:2454] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 1.
I20260811 02:58:02.758481 24403 leader_election.cc:302] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b3230fa47e4bb4b004cc3d04e10b94, e1f4bea25f90448ea95ff41a06a0e267; no voters:
I20260811 02:58:02.758653 24713 raft_consensus.cc:2781] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:02.758742 24713 raft_consensus.cc:686] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 LEADER]: Becoming Leader. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Running, Role: LEADER
I20260811 02:58:02.758873 24713 consensus_queue.cc:237] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.760478 23688 catalog_manager.cc:5495] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67). New cstate: current_term: 1 leader_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:02.809630 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:02.806641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.8.203.67:42251)
0811 02:58:02.806775 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:02.806780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:02.806795 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:58:02.806931 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:02.806934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:02.806980 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:02.807041 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:58:02.807298 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.807804 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.808631 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:02.808846 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:02.808934 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:02.809191 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:58:02.809267 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:58:02.809357 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:02.809441 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 02:58:02.809493 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:58:02.811491 24263 raft_consensus.cc:1261] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Refusing update from remote peer c1b3230fa47e4bb4b004cc3d04e10b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:02.811491 23989 raft_consensus.cc:1261] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Refusing update from remote peer c1b3230fa47e4bb4b004cc3d04e10b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:02.811908 24741 consensus_queue.cc:1035] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.812070 24713 consensus_queue.cc:1035] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.818898 23688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:02.819027 23688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:02.820817 23688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Sending DeleteTablet for 3 replicas of tablet 04e2d52d46ab4762b513faecd53a9b5f
I20260811 02:58:02.821344 23949 tablet_service.cc:1514] Processing DeleteTablet for tablet 04e2d52d46ab4762b513faecd53a9b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:50200
I20260811 02:58:02.821405 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet 04e2d52d46ab4762b513faecd53a9b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:56278
I20260811 02:58:02.821524 24805 tablet_replica.cc:330] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: stopping tablet replica
I20260811 02:58:02.821529 24481 tablet_service.cc:1514] Processing DeleteTablet for tablet 04e2d52d46ab4762b513faecd53a9b5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52148
I20260811 02:58:02.821574 24806 tablet_replica.cc:330] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: stopping tablet replica
I20260811 02:58:02.821652 24805 raft_consensus.cc:2229] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:02.821669 24808 tablet_replica.cc:330] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: stopping tablet replica
I20260811 02:58:02.821684 24806 raft_consensus.cc:2229] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:02.821767 24808 raft_consensus.cc:2229] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:58:02.821780 24805 raft_consensus.cc:2258] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:02.821796 24806 raft_consensus.cc:2258] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:02.821931 24808 raft_consensus.cc:2258] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:02.822402 24805 ts_tablet_manager.cc:1905] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:02.822631 24806 ts_tablet_manager.cc:1905] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:02.822746 24808 ts_tablet_manager.cc:1905] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:02.824548 24806 ts_tablet_manager.cc:1918] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:02.824617 24806 log.cc:1196] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/04e2d52d46ab4762b513faecd53a9b5f
I20260811 02:58:02.824819 24805 ts_tablet_manager.cc:1918] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:02.824853 24806 ts_tablet_manager.cc:1939] T 04e2d52d46ab4762b513faecd53a9b5f P e1f4bea25f90448ea95ff41a06a0e267: Deleting consensus metadata
I20260811 02:58:02.824918 24805 log.cc:1196] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/04e2d52d46ab4762b513faecd53a9b5f
I20260811 02:58:02.824950 24808 ts_tablet_manager.cc:1918] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:02.825018 24808 log.cc:1196] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/04e2d52d46ab4762b513faecd53a9b5f
I20260811 02:58:02.825161 24805 ts_tablet_manager.cc:1939] T 04e2d52d46ab4762b513faecd53a9b5f P 707bc626bbd948e1b422f4c907be3cf8: Deleting consensus metadata
I20260811 02:58:02.825239 24808 ts_tablet_manager.cc:1939] T 04e2d52d46ab4762b513faecd53a9b5f P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting consensus metadata
I20260811 02:58:02.825512 23656 catalog_manager.cc:4841] TS e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795): tablet 04e2d52d46ab4762b513faecd53a9b5f (table test-table [id=c5c4e6973a31430fa60103792df6a46b]) successfully deleted
I20260811 02:58:02.825793 23656 catalog_manager.cc:4841] TS c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251): tablet 04e2d52d46ab4762b513faecd53a9b5f (table test-table [id=c5c4e6973a31430fa60103792df6a46b]) successfully deleted
I20260811 02:58:02.825883 23657 catalog_manager.cc:4841] TS 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063): tablet 04e2d52d46ab4762b513faecd53a9b5f (table test-table [id=c5c4e6973a31430fa60103792df6a46b]) successfully deleted
I20260811 02:58:02.826134 23688 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:58:02.826491 23688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:58:02.830576 23949 tablet_service.cc:1467] Processing CreateTablet for tablet a9992974d5a54595a6265752c30c4f5f (DEFAULT_TABLE table=test-table [id=405d85ed4d0d44b48f03ea90262fbf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:02.830824 23949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9992974d5a54595a6265752c30c4f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:02.830783 24227 tablet_service.cc:1467] Processing CreateTablet for tablet a9992974d5a54595a6265752c30c4f5f (DEFAULT_TABLE table=test-table [id=405d85ed4d0d44b48f03ea90262fbf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:02.830849 24481 tablet_service.cc:1467] Processing CreateTablet for tablet a9992974d5a54595a6265752c30c4f5f (DEFAULT_TABLE table=test-table [id=405d85ed4d0d44b48f03ea90262fbf94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:02.830996 24227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9992974d5a54595a6265752c30c4f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:02.831115 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9992974d5a54595a6265752c30c4f5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:02.832711 24859 tablet_bootstrap.cc:492] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: Bootstrap starting.
I20260811 02:58:02.833012 24856 tablet_bootstrap.cc:492] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8: Bootstrap starting.
I20260811 02:58:02.833432 24854 tablet_bootstrap.cc:492] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: Bootstrap starting.
I20260811 02:58:02.833590 24859 tablet_bootstrap.cc:654] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:02.833766 24856 tablet_bootstrap.cc:654] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:02.834237 24854 tablet_bootstrap.cc:654] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:02.834658 24859 tablet_bootstrap.cc:492] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: No bootstrap required, opened a new log
I20260811 02:58:02.834738 24859 ts_tablet_manager.cc:1397] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:02.835125 24859 raft_consensus.cc:348] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.835242 24859 raft_consensus.cc:374] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:02.835294 24859 raft_consensus.cc:729] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Initialized, Role: FOLLOWER
I20260811 02:58:02.835384 24854 tablet_bootstrap.cc:492] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: No bootstrap required, opened a new log
I20260811 02:58:02.835465 24854 ts_tablet_manager.cc:1397] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:02.835422 24859 consensus_queue.cc:260] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.835682 24859 ts_tablet_manager.cc:1428] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:58:02.835842 24854 raft_consensus.cc:348] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.835968 24854 raft_consensus.cc:374] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:02.836005 24854 raft_consensus.cc:729] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f4bea25f90448ea95ff41a06a0e267, State: Initialized, Role: FOLLOWER
I20260811 02:58:02.836123 24854 consensus_queue.cc:260] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.836326 24856 tablet_bootstrap.cc:492] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8: No bootstrap required, opened a new log
I20260811 02:58:02.836362 24854 ts_tablet_manager.cc:1428] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:58:02.836400 24856 ts_tablet_manager.cc:1397] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:58:02.836771 24856 raft_consensus.cc:348] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.837031 24856 raft_consensus.cc:374] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:02.837077 24856 raft_consensus.cc:729] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707bc626bbd948e1b422f4c907be3cf8, State: Initialized, Role: FOLLOWER
I20260811 02:58:02.837195 24856 consensus_queue.cc:260] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.837478 24856 ts_tablet_manager.cc:1428] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:58:02.930900 24669 raft_consensus.cc:482] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:02.931023 24669 raft_consensus.cc:504] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.931351 24669 leader_election.cc:288] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795)
I20260811 02:58:02.931700 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9992974d5a54595a6265752c30c4f5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8" is_pre_election: true
I20260811 02:58:02.931712 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9992974d5a54595a6265752c30c4f5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267" is_pre_election: true
I20260811 02:58:02.931838 23989 raft_consensus.cc:2454] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 0.
I20260811 02:58:02.931845 24263 raft_consensus.cc:2454] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 0.
I20260811 02:58:02.932144 24406 leader_election.cc:302] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, c1b3230fa47e4bb4b004cc3d04e10b94, e1f4bea25f90448ea95ff41a06a0e267; no voters:
I20260811 02:58:02.932323 24669 raft_consensus.cc:2781] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:02.932385 24669 raft_consensus.cc:482] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:02.932413 24669 raft_consensus.cc:3037] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.933423 24669 raft_consensus.cc:504] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.933732 24669 leader_election.cc:288] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 election: Requested vote from peers 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063), e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795)
I20260811 02:58:02.934023 23989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9992974d5a54595a6265752c30c4f5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707bc626bbd948e1b422f4c907be3cf8"
I20260811 02:58:02.934082 24263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9992974d5a54595a6265752c30c4f5f" candidate_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1f4bea25f90448ea95ff41a06a0e267"
I20260811 02:58:02.934137 23989 raft_consensus.cc:3037] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.934199 24263 raft_consensus.cc:3037] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:02.935034 24263 raft_consensus.cc:2454] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 1.
I20260811 02:58:02.935034 23989 raft_consensus.cc:2454] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1b3230fa47e4bb4b004cc3d04e10b94 in term 1.
I20260811 02:58:02.935282 24406 leader_election.cc:302] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707bc626bbd948e1b422f4c907be3cf8, c1b3230fa47e4bb4b004cc3d04e10b94; no voters:
I20260811 02:58:02.935453 24669 raft_consensus.cc:2781] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:02.935549 24669 raft_consensus.cc:686] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 LEADER]: Becoming Leader. State: Replica: c1b3230fa47e4bb4b004cc3d04e10b94, State: Running, Role: LEADER
I20260811 02:58:02.935688 24669 consensus_queue.cc:237] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } }
I20260811 02:58:02.937250 23686 catalog_manager.cc:5495] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67). New cstate: current_term: 1 leader_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b3230fa47e4bb4b004cc3d04e10b94" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 42251 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:02.966295 24263 raft_consensus.cc:1261] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Refusing update from remote peer c1b3230fa47e4bb4b004cc3d04e10b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:02.966295 23989 raft_consensus.cc:1261] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Refusing update from remote peer c1b3230fa47e4bb4b004cc3d04e10b94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:02.966666 24713 consensus_queue.cc:1035] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "707bc626bbd948e1b422f4c907be3cf8" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.966804 24669 consensus_queue.cc:1035] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1f4bea25f90448ea95ff41a06a0e267" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:02.997853 23686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:02.997993 23686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:02.999691 23686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 092ba12eb7ae4cc99d84532bd04db4ec: Sending DeleteTablet for 3 replicas of tablet a9992974d5a54595a6265752c30c4f5f
I20260811 02:58:03.000164 23949 tablet_service.cc:1514] Processing DeleteTablet for tablet a9992974d5a54595a6265752c30c4f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:50200
I20260811 02:58:03.000274 24227 tablet_service.cc:1514] Processing DeleteTablet for tablet a9992974d5a54595a6265752c30c4f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:56278
I20260811 02:58:03.000335 24805 tablet_replica.cc:330] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8: stopping tablet replica
I20260811 02:58:03.000344 24481 tablet_service.cc:1514] Processing DeleteTablet for tablet a9992974d5a54595a6265752c30c4f5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:02 UTC) from {username='oryx', principal='oryx/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:52148
I20260811 02:58:03.000481 24806 tablet_replica.cc:330] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: stopping tablet replica
I20260811 02:58:03.000469 24805 raft_consensus.cc:2229] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:03.000485 24808 tablet_replica.cc:330] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: stopping tablet replica
I20260811 02:58:03.000574 24805 raft_consensus.cc:2258] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:03.000587 24808 raft_consensus.cc:2229] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:58:03.000589 24806 raft_consensus.cc:2229] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:03.000674 24806 raft_consensus.cc:2258] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:03.000744 24808 raft_consensus.cc:2258] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:03.001430 24805 ts_tablet_manager.cc:1905] T a9992974d5a54595a6265752c30c4f5f P 707bc626bbd948e1b422f4c907be3cf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:03.001519 24808 ts_tablet_manager.cc:1905] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:03.001583 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 23827
I20260811 02:58:03.002419 24806 ts_tablet_manager.cc:1905] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:03.003913 24808 ts_tablet_manager.cc:1918] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:58:03.004060 24808 log.cc:1196] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/a9992974d5a54595a6265752c30c4f5f
I20260811 02:58:03.004312 24808 ts_tablet_manager.cc:1939] T a9992974d5a54595a6265752c30c4f5f P c1b3230fa47e4bb4b004cc3d04e10b94: Deleting consensus metadata
I20260811 02:58:03.004861 24806 ts_tablet_manager.cc:1918] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:58:03.004931 24806 log.cc:1196] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/a9992974d5a54595a6265752c30c4f5f
I20260811 02:58:03.004997 23656 catalog_manager.cc:4841] TS c1b3230fa47e4bb4b004cc3d04e10b94 (127.8.203.67:42251): tablet a9992974d5a54595a6265752c30c4f5f (table test-table [id=405d85ed4d0d44b48f03ea90262fbf94]) successfully deleted
I20260811 02:58:03.005138 24806 ts_tablet_manager.cc:1939] T a9992974d5a54595a6265752c30c4f5f P e1f4bea25f90448ea95ff41a06a0e267: Deleting consensus metadata
I20260811 02:58:03.005740 23656 catalog_manager.cc:4841] TS e1f4bea25f90448ea95ff41a06a0e267 (127.8.203.66:38795): tablet a9992974d5a54595a6265752c30c4f5f (table test-table [id=405d85ed4d0d44b48f03ea90262fbf94]) successfully deleted
W20260811 02:58:03.018985 23657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:40063 (error 108)
I20260811 02:58:03.019290 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 24101
I20260811 02:58:03.034075 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:03.032586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40063 (local address 127.0.0.1:50218)
0811 02:58:03.033881 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:03.033967 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:40063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1212,"thread_start_us":944,"threads_started":1}
W20260811 02:58:03.034183 23657 catalog_manager.cc:4584] TS 707bc626bbd948e1b422f4c907be3cf8 (127.8.203.65:40063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a9992974d5a54595a6265752c30c4f5f: Network error: Client connection negotiation failed: client connection to 127.8.203.65:40063: connect: Connection refused (error 111)
I20260811 02:58:03.041576 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 24375
I20260811 02:58:03.054242 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 23620
2026-08-11T02:58:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4320 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:58:03 dist-test-slave-gjw7 krb5kdc[24892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:58:03 dist-test-slave-gjw7 krb5kdc[24892](info): set up 2 sockets
Aug 11 02:58:03 dist-test-slave-gjw7 krb5kdc[24892](info): commencing operation
krb5kdc: starting...
W20260811 02:58:04.255154 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:58:04 dist-test-slave-gjw7 krb5kdc[24892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:58:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:58:04.369659 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:44375
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:44375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:04.478018 24914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:04.478250 24914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:04.478303 24914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:04.482161 24914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:04.482208 24914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:04.482234 24914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:04.482255 24914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:04.482283 24914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:04.487311 24914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:44375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:44375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:04.488224 24914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:04.489146 24914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:04.497052 24927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:04.497097 24914 server_base.cc:1034] running on GCE node
W20260811 02:58:04.497035 24928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:04.497051 24933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:04.497766 24914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:04.498764 24914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:04.499917 24914 hybrid_clock.cc:648] HybridClock initialized: now 1786417084499904 us; error 25 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T02:58:04Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1456 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:58:04 dist-test-slave-gjw7 krb5kdc[24939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:58:04 dist-test-slave-gjw7 krb5kdc[24939](info): set up 2 sockets
Aug 11 02:58:04 dist-test-slave-gjw7 krb5kdc[24939](info): commencing operation
krb5kdc: starting...
W20260811 02:58:05.708204 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:58:05 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:58:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:58:05.819789 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45705
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:45705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:05.930473 24964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:05.930691 24964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:05.930727 24964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:05.934559 24964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:05.934604 24964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:05.934621 24964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:05.934638 24964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:05.934669 24964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:05.939611 24964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:45705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:05.940517 24964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:05.941481 24964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:05.949494 24986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:05.949496 24978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:05.949506 24979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:05.949625 24964 server_base.cc:1034] running on GCE node
I20260811 02:58:05.950317 24964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:05.951356 24964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:05.952512 24964 hybrid_clock.cc:648] HybridClock initialized: now 1786417085952499 us; error 28 us; skew 500 ppm
Aug 11 02:58:05 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417085, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:05.955120 24964 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:58:05.955945 24964 webserver.cc:460] Webserver started at http://127.8.203.126:41865/ using document root <none> and password file <none>
I20260811 02:58:05.956475 24964 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:05.956539 24964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:05.956743 24964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:05.958726 24964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "046a3e3167684e0ca214f6ea57893661"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-gjw7"
server_key: "33c4a93c449d2a89d13199a7acbc2a0b"
server_key_iv: "b49e0cf53e202c4c47a0190e5506b628"
server_key_version: "encryptionkey@0"
I20260811 02:58:05.959249 24964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "046a3e3167684e0ca214f6ea57893661"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-gjw7"
server_key: "33c4a93c449d2a89d13199a7acbc2a0b"
server_key_iv: "b49e0cf53e202c4c47a0190e5506b628"
server_key_version: "encryptionkey@0"
I20260811 02:58:05.962757 24964 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:58:05.965004 24999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:05.966150 24964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:05.966261 24964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "046a3e3167684e0ca214f6ea57893661"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-gjw7"
server_key: "33c4a93c449d2a89d13199a7acbc2a0b"
server_key_iv: "b49e0cf53e202c4c47a0190e5506b628"
server_key_version: "encryptionkey@0"
I20260811 02:58:05.966359 24964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:05.987692 24964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:05.990537 24964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:05.990720 24964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:05.997444 24964 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:45705
I20260811 02:58:05.997460 25103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:45705 every 8 connection(s)
I20260811 02:58:05.998379 24964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:06.001065 25106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:06.004972 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 24964
I20260811 02:58:06.005075 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:58:06.005362 9005 external_mini_cluster.cc:1428] Setting key 19ee83166eb700a3fb1bb38d86960021
I20260811 02:58:06.006693 25106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Bootstrap starting.
I20260811 02:58:06.008878 25106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:06.009615 25106 log.cc:824] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:06.011351 25106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: No bootstrap required, opened a new log
Aug 11 02:58:06 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:06.015211 25106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:06.015458 25106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:06.015518 25106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046a3e3167684e0ca214f6ea57893661, State: Initialized, Role: FOLLOWER
I20260811 02:58:06.015980 25106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:06.016104 25106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:06.016162 25106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:06.016256 25106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:06.017016 25106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:06.017352 25106 leader_election.cc:302] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 046a3e3167684e0ca214f6ea57893661; no voters:
I20260811 02:58:06.017675 25106 leader_election.cc:288] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:06.017755 25115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:06.017947 25115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 LEADER]: Becoming Leader. State: Replica: 046a3e3167684e0ca214f6ea57893661, State: Running, Role: LEADER
I20260811 02:58:06.018287 25115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:06.019001 25106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:06.019488 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:06.006732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.8.203.126:45705)
0811 02:58:06.007149 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:06.007165 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:06.007202 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:58:06.007822 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:06.007840 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:06.008158 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:06.008470 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:06.009323 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.010368 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.011203 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.011708 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.011854 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:06.014562 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:06.014567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:06.014580 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:06.014606 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:06.016279 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.016710 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.016717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.016723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.016841 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.017096 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.017100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.017102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.017583 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:06.017762 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:06.018415 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 02:58:06.018678 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":122,"threads_started":1}
I20260811 02:58:06.019893 25117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "046a3e3167684e0ca214f6ea57893661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } } }
I20260811 02:58:06.020015 25117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:06.020295 25116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 046a3e3167684e0ca214f6ea57893661. Latest consensus state: current_term: 1 leader_uuid: "046a3e3167684e0ca214f6ea57893661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } } }
I20260811 02:58:06.020378 25116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:06.020398 25135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:06.023257 25135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:06.028362 25135 catalog_manager.cc:1347] Generated new cluster ID: 684c6be653684ebf8a1182df41d5bd60
I20260811 02:58:06.028453 25135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:06.051071 25135 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:58:06.052350 25135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:06.071100 25135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Generated new TSK 0
I20260811 02:58:06.071813 25135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:58:06.101752 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45705
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:06.238600 25156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:06.238837 25156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:06.238881 25156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:06.242844 25156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:06.242894 25156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:06.242960 25156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:58:06.248116 25156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:06.249114 25156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:06.250155 25156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:06.259410 25170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:06.259445 25163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:06.259629 25156 server_base.cc:1034] running on GCE node
W20260811 02:58:06.259529 25167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:06.260089 25156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:06.260682 25156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:06.261849 25156 hybrid_clock.cc:648] HybridClock initialized: now 1786417086261831 us; error 27 us; skew 500 ppm
Aug 11 02:58:06 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417086, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:06.264374 25156 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:58:06.265149 25156 webserver.cc:460] Webserver started at http://127.8.203.65:46213/ using document root <none> and password file <none>
I20260811 02:58:06.265695 25156 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:06.265758 25156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:06.265954 25156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:06.267935 25156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "673d232d9d9e47488a77c6e767b4ba45"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "0855d98cd2c767aa710e2afd2b61f845"
server_key_iv: "3bde69e6822b9e7300c4226083773430"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.268473 25156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "673d232d9d9e47488a77c6e767b4ba45"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "0855d98cd2c767aa710e2afd2b61f845"
server_key_iv: "3bde69e6822b9e7300c4226083773430"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.272034 25156 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 02:58:06.274334 25185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.275454 25156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:06.275552 25156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "673d232d9d9e47488a77c6e767b4ba45"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "0855d98cd2c767aa710e2afd2b61f845"
server_key_iv: "3bde69e6822b9e7300c4226083773430"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.275650 25156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:06.291712 25156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:06.294409 25156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:06.294560 25156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:06.295066 25156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:06.295809 25156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:06.295855 25156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.295897 25156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:06.295935 25156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.303864 25156 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:36845
I20260811 02:58:06.303896 25428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:36845 every 8 connection(s)
I20260811 02:58:06.304659 25156 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:06.308306 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 25156
I20260811 02:58:06.308422 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:06.308715 9005 external_mini_cluster.cc:1428] Setting key 227ff3a6f8ed4d805b2400d7014bd26f
Aug 11 02:58:06 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417086, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:06.317530 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:06.306280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:35823 (local address 127.8.203.126:45705)
0811 02:58:06.306414 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:06.306418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:06.307197 (+ 779us) server_negotiation.cc:407] Connection header received
0811 02:58:06.308413 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:06.308417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:06.308474 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:06.308559 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:06.309710 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.310417 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.311498 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.311801 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.311903 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:06.314336 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:06.314340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:06.314343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:06.314369 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:06.315626 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.316100 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.316103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.316104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.316180 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.316486 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.316489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.316491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.316665 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:06.316751 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:06.317213 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 02:58:06.317365 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":46}
I20260811 02:58:06.318238 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:06.306639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:35823)
0811 02:58:06.307064 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:06.307104 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:06.308209 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:06.308698 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:06.308704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:06.309046 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:06.309575 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:06.309594 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.310537 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.310540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:06.311381 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:06.311390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.311927 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.311930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:06.312106 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:06.312644 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:06.312663 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:06.314188 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:06.315731 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:06.315737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:06.315750 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:06.316009 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:06.316258 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:06.316261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:06.316264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:06.316401 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:06.316761 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:06.316768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:06.316994 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 02:58:06.317152 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 02:58:06.317491 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:58:06.319343 25430 heartbeater.cc:347] Connected to a master server at 127.8.203.126:45705
I20260811 02:58:06.319550 25430 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:06.320107 25430 heartbeater.cc:511] Master 127.8.203.126:45705 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 02:58:06.321604 25031 ts_manager.cc:194] Registered new tserver with Master: 673d232d9d9e47488a77c6e767b4ba45 (127.8.203.65:36845)
I20260811 02:58:06.322685 25031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:35823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:58:06.335354 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45705
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:06.442819 25442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:06.443054 25442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:06.443140 25442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:06.447026 25442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:06.447074 25442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:06.447127 25442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:58:06.451987 25442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:06.452862 25442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:06.453847 25442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:06.462561 25450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:06.462534 25448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:06.462594 25442 server_base.cc:1034] running on GCE node
W20260811 02:58:06.462548 25452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:06.463129 25442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:06.463690 25442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:06.464864 25442 hybrid_clock.cc:648] HybridClock initialized: now 1786417086464847 us; error 34 us; skew 500 ppm
Aug 11 02:58:06 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417086, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:06.467300 25442 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:58:06.468092 25442 webserver.cc:460] Webserver started at http://127.8.203.66:40293/ using document root <none> and password file <none>
I20260811 02:58:06.468581 25442 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:06.468645 25442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:06.468842 25442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:06.470899 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "aa076a47dc76443dbad266657ac5a8a2"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "d0d81551f2088a20d4ef8c79c8cab718"
server_key_iv: "fbcbc1b9571bf80bfcf8762baeb544a4"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.471414 25442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "aa076a47dc76443dbad266657ac5a8a2"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "d0d81551f2088a20d4ef8c79c8cab718"
server_key_iv: "fbcbc1b9571bf80bfcf8762baeb544a4"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.474846 25442 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:58:06.477036 25476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.478163 25442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:06.478266 25442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "aa076a47dc76443dbad266657ac5a8a2"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "d0d81551f2088a20d4ef8c79c8cab718"
server_key_iv: "fbcbc1b9571bf80bfcf8762baeb544a4"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.478353 25442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:06.489130 25442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:06.491526 25442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:06.491676 25442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:06.492175 25442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:06.492954 25442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:06.493002 25442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.493052 25442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:06.493078 25442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.501211 25442 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:46597
I20260811 02:58:06.501237 25719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:46597 every 8 connection(s)
I20260811 02:58:06.501993 25442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:58:06 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417086, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:06.510540 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 25442
I20260811 02:58:06.510653 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:58:06.510900 9005 external_mini_cluster.cc:1428] Setting key faf23f7bd822a00afec5a653e2e09d32
I20260811 02:58:06.514499 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:06.503589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:56929 (local address 127.8.203.126:45705)
0811 02:58:06.503726 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:06.503730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:06.504509 (+ 779us) server_negotiation.cc:407] Connection header received
0811 02:58:06.505746 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:06.505750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:06.505808 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:06.505883 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:06.506938 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.507469 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.508549 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.508778 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.508866 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:06.511089 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:06.511094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:06.511096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:06.511124 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:06.512613 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.513066 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.513070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.513072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.513162 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.513478 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.513481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.513483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.513685 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:06.513782 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:06.514216 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 02:58:06.514347 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:58:06.515329 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:06.503951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:56929)
0811 02:58:06.504369 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:06.504410 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:06.505542 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:06.506049 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:06.506056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:06.506365 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:06.506800 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:06.506813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.507586 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.507589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:06.508434 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:06.508444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.508912 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.508916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:06.509066 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:06.509537 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:06.509553 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:06.510929 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:06.512719 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:06.512723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:06.512734 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:06.512978 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:06.513242 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:06.513244 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:06.513246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:06.513399 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:06.513780 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:06.513785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:06.513998 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 02:58:06.514133 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 02:58:06.514429 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":118,"threads_started":1}
I20260811 02:58:06.516378 25720 heartbeater.cc:347] Connected to a master server at 127.8.203.126:45705
I20260811 02:58:06.516577 25720 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:06.516999 25720 heartbeater.cc:511] Master 127.8.203.126:45705 requested a full tablet report, sending...
I20260811 02:58:06.518059 25031 ts_manager.cc:194] Registered new tserver with Master: aa076a47dc76443dbad266657ac5a8a2 (127.8.203.66:46597)
I20260811 02:58:06.518648 25031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:56929
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:58:06.537380 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45705
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:06.647681 25731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:06.647921 25731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:06.647964 25731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:06.651870 25731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:06.651923 25731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:06.651990 25731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:58:06.656963 25731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:06.657876 25731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:06.658794 25731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:06.667810 25745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:06.667809 25738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:06.667948 25731 server_base.cc:1034] running on GCE node
W20260811 02:58:06.668084 25739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:06.668469 25731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:06.669060 25731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:06.670228 25731 hybrid_clock.cc:648] HybridClock initialized: now 1786417086670209 us; error 29 us; skew 500 ppm
Aug 11 02:58:06 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417086, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:06.672770 25731 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:58:06.673591 25731 webserver.cc:460] Webserver started at http://127.8.203.67:41997/ using document root <none> and password file <none>
I20260811 02:58:06.674099 25731 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:06.674158 25731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:06.674352 25731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:06.676232 25731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "f162607cc1e345f5b5767cad175b8a5a"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "a306018602ecf03fbe299fdb299e3b68"
server_key_iv: "fc5c5cf2300ae27c4ecf4838ca3de9b3"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.676749 25731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "f162607cc1e345f5b5767cad175b8a5a"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "a306018602ecf03fbe299fdb299e3b68"
server_key_iv: "fc5c5cf2300ae27c4ecf4838ca3de9b3"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.680284 25731 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:58:06.682518 25756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.683627 25731 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:06.683729 25731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "f162607cc1e345f5b5767cad175b8a5a"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "a306018602ecf03fbe299fdb299e3b68"
server_key_iv: "fc5c5cf2300ae27c4ecf4838ca3de9b3"
server_key_version: "encryptionkey@0"
I20260811 02:58:06.683816 25731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:06.701668 25731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:06.704142 25731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:06.704298 25731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:06.704821 25731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:06.705621 25731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:06.705668 25731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.705717 25731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:06.705746 25731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:06.713815 25731 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:39117
I20260811 02:58:06.713840 26002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:39117 every 8 connection(s)
I20260811 02:58:06.714615 25731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
Aug 11 02:58:06 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417086, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:06.723470 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 25731
I20260811 02:58:06.723600 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:58:06.723894 9005 external_mini_cluster.cc:1428] Setting key 892c2bac28c6da159403b5f103b41142
I20260811 02:58:06.727002 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:06.716220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:37415 (local address 127.8.203.126:45705)
0811 02:58:06.716358 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:06.716363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:06.717132 (+ 769us) server_negotiation.cc:407] Connection header received
0811 02:58:06.718357 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:06.718360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:06.718406 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:06.718475 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:06.719607 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.720172 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.721250 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.721496 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.721569 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:06.723817 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:06.723820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:06.723823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:06.723852 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:06.725284 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.725744 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.725748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.725749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.725842 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:06.726173 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:06.726175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:06.726176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:06.726350 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:06.726421 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:06.726796 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 02:58:06.726890 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:58:06.727694 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:06.716583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:37415)
0811 02:58:06.716993 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:06.717033 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:06.718172 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:06.718654 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:06.718661 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:06.718997 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:06.719490 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:06.719506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.720270 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.720273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:06.721148 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:06.721158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:06.721599 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:06.721602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:06.721740 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:06.722233 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:06.722249 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:06.723657 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:06.725410 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:06.725415 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:06.725426 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:06.725655 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:06.725936 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:06.725939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:06.725941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:06.726100 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:06.726424 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:06.726429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:06.726625 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 02:58:06.726765 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 02:58:06.726995 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":118,"threads_started":1}
I20260811 02:58:06.728619 26003 heartbeater.cc:347] Connected to a master server at 127.8.203.126:45705
I20260811 02:58:06.728817 26003 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:06.729238 26003 heartbeater.cc:511] Master 127.8.203.126:45705 requested a full tablet report, sending...
I20260811 02:58:06.730170 25031 ts_manager.cc:194] Registered new tserver with Master: f162607cc1e345f5b5767cad175b8a5a (127.8.203.67:39117)
I20260811 02:58:06.730690 25031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:37415
I20260811 02:58:06.737417 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab.
Entry for principal oryx/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab.
WARNING: no policy specified for oryx/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab.
Entry for principal oryx/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab.
WARNING: no policy specified for oryx/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab.
Entry for principal oryx/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab.
I20260811 02:58:06.801869 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 25156
I20260811 02:58:06.824028 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 25442
I20260811 02:58:06.839613 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 25731
I20260811 02:58:06.860534 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 24964
I20260811 02:58:06.877143 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45705
--webserver_interface=127.8.203.126
--webserver_port=41865
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:45705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:06.986097 26018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:06.986332 26018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:06.986384 26018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:06.990257 26018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:06.990315 26018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:06.990340 26018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:06.990362 26018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:06.990391 26018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:06.995558 26018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:45705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=41865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:06.996569 26018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:06.997579 26018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:07.005596 26033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:07.005582 26038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:07.005574 26032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:07.005630 26018 server_base.cc:1034] running on GCE node
I20260811 02:58:07.006353 26018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:07.007383 26018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:07.008543 26018 hybrid_clock.cc:648] HybridClock initialized: now 1786417087008522 us; error 29 us; skew 500 ppm
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417087, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:07.011101 26018 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:58:07.011950 26018 webserver.cc:460] Webserver started at http://127.8.203.126:41865/ using document root <none> and password file <none>
I20260811 02:58:07.012472 26018 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:07.012552 26018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:07.016383 26018 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:58:07.018435 26053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.019570 26018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:07.019675 26018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "046a3e3167684e0ca214f6ea57893661"
format_stamp: "Formatted at 2026-08-11 02:58:05 on dist-test-slave-gjw7"
server_key: "33c4a93c449d2a89d13199a7acbc2a0b"
server_key_iv: "b49e0cf53e202c4c47a0190e5506b628"
server_key_version: "encryptionkey@0"
I20260811 02:58:07.020192 26018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:07.044694 26018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:07.047494 26018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:07.047672 26018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:07.054417 26018 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:45705
I20260811 02:58:07.054436 26164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:45705 every 8 connection(s)
I20260811 02:58:07.055454 26018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:07.059553 26168 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:58:07.061105 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Bootstrap starting.
I20260811 02:58:07.063033 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 26018
I20260811 02:58:07.063512 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:36845
--local_ip_for_outbound_sockets=127.8.203.65
--tserver_master_addrs=127.8.203.126:45705
--webserver_port=46213
--webserver_interface=127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:58:07.078087 26168 log.cc:824] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:07.084448 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:58:07.084757 26168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Bootstrap complete.
I20260811 02:58:07.088796 26168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:07.089190 26168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046a3e3167684e0ca214f6ea57893661, State: Initialized, Role: FOLLOWER
I20260811 02:58:07.089701 26168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:07.089830 26168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:07.089905 26168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:07.090005 26168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:58:07.091804 26168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:07.092113 26168 leader_election.cc:302] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 046a3e3167684e0ca214f6ea57893661; no voters:
I20260811 02:58:07.092399 26168 leader_election.cc:288] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:58:07.092496 26175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:58:07.092691 26175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [term 2 LEADER]: Becoming Leader. State: Replica: 046a3e3167684e0ca214f6ea57893661, State: Running, Role: LEADER
I20260811 02:58:07.092971 26175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } }
I20260811 02:58:07.093618 26168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:07.093958 26178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 046a3e3167684e0ca214f6ea57893661. Latest consensus state: current_term: 2 leader_uuid: "046a3e3167684e0ca214f6ea57893661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } } }
I20260811 02:58:07.094079 26178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:07.094311 26177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "046a3e3167684e0ca214f6ea57893661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046a3e3167684e0ca214f6ea57893661" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45705 } } }
I20260811 02:58:07.094424 26177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661 [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:07.095677 26184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:07.098660 26184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:07.099273 26184 catalog_manager.cc:1259] Loaded cluster ID: 684c6be653684ebf8a1182df41d5bd60
I20260811 02:58:07.099334 26184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:07.101637 26184 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:07.102268 26184 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 046a3e3167684e0ca214f6ea57893661: Loaded TSK: 0
I20260811 02:58:07.103436 26184 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:58:07.175169 26172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:07.175407 26172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:07.175465 26172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:07.179378 26172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:07.179432 26172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:07.179508 26172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:58:07.184487 26172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.65.keytab
--principal=oryx/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:36845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=46213
--tserver_master_addrs=127.8.203.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:07.185431 26172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:07.186354 26172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:07.196586 26217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:07.196586 26218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:07.196609 26222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:07.197067 26172 server_base.cc:1034] running on GCE node
I20260811 02:58:07.197587 26172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:07.198379 26172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:07.199591 26172 hybrid_clock.cc:648] HybridClock initialized: now 1786417087199563 us; error 43 us; skew 500 ppm
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417087, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:07.202920 26172 init.cc:377] Logged in from keytab as oryx/127.8.203.65@KRBTEST.COM (short username oryx)
I20260811 02:58:07.203715 26172 webserver.cc:460] Webserver started at http://127.8.203.65:46213/ using document root <none> and password file <none>
I20260811 02:58:07.204226 26172 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:07.204299 26172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:07.208096 26172 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:07.210228 26241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.211463 26172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:07.211577 26172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "673d232d9d9e47488a77c6e767b4ba45"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "0855d98cd2c767aa710e2afd2b61f845"
server_key_iv: "3bde69e6822b9e7300c4226083773430"
server_key_version: "encryptionkey@0"
I20260811 02:58:07.212066 26172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:07.223228 26172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:07.226856 26172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:07.227065 26172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:07.227715 26172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:07.228699 26172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:07.228765 26172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.228826 26172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:07.228859 26172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.239827 26172 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:36845
I20260811 02:58:07.239879 26465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:36845 every 8 connection(s)
I20260811 02:58:07.240803 26172 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:07.250522 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 26172
I20260811 02:58:07.250970 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:46597
--local_ip_for_outbound_sockets=127.8.203.66
--tserver_master_addrs=127.8.203.126:45705
--webserver_port=40293
--webserver_interface=127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:07.253579 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.242856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:43151)
0811 02:58:07.243310 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.243358 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.244513 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.245692 (+ 1179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.245700 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.246092 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.246541 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.246555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.247808 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.247811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.248741 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.248752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.249531 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.249535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.249709 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.250181 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.250197 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.252596 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":141,"threads_started":1}
W20260811 02:58:07.253571 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.242719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:43151 (local address 127.8.203.126:45705)
0811 02:58:07.243175 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.243196 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.243461 (+ 265us) server_negotiation.cc:407] Connection header received
0811 02:58:07.244918 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.244931 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.245223 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.245592 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.246721 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.247675 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.248875 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.249377 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.249527 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.252827 (+ 3300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:43151: BlockingRecv error: recv got EOF from 127.8.203.65:43151 (error 108)
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":137,"threads_started":1}
W20260811 02:58:07.254230 26468 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:45705 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.203.126:45705: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:07.259631 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.254659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:57491)
0811 02:58:07.254761 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.254776 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.254853 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.255060 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.255062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.255210 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.255302 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.255306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.256282 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.256284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.256822 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.256831 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.257365 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.257368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.257435 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.257844 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.257855 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.259506 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 02:58:07.259721 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.254747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:57491 (local address 127.8.203.126:45705)
0811 02:58:07.254867 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.254871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.254886 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:58:07.254922 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.254925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.254967 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.255042 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.255405 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.256175 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.256928 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.257246 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.257367 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.259623 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:57491: BlockingRecv error: recv got EOF from 127.8.203.65:57491 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:07.264823 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.260124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:48479)
0811 02:58:07.260247 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.260261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.260332 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.260532 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.260534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.260651 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.260720 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.260724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.261702 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.261704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.262203 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.262211 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.262728 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.262730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.262781 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.263158 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.263169 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.264707 (+ 1538us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 02:58:07.265007 26468 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:58:07.264905 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.260229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:48479 (local address 127.8.203.126:45705)
0811 02:58:07.260344 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.260348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.260363 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:58:07.260399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.260401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.260442 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.260525 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.260817 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.261578 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.262312 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.262631 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.262739 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.264816 (+ 2077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:48479: BlockingRecv error: recv got EOF from 127.8.203.65:48479 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 02:58:07.362161 26476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:07.362409 26476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:07.362455 26476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:07.366391 26476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:07.366439 26476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:07.366492 26476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:58:07.371429 26476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.66.keytab
--principal=oryx/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:46597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=40293
--tserver_master_addrs=127.8.203.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:07.372320 26476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:07.373240 26476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:07.382344 26497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:07.382355 26488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:07.382758 26476 server_base.cc:1034] running on GCE node
W20260811 02:58:07.383239 26490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:07.383618 26476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:07.384222 26476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:07.385346 26476 hybrid_clock.cc:648] HybridClock initialized: now 1786417087385329 us; error 33 us; skew 500 ppm
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417087, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:07.387872 26476 init.cc:377] Logged in from keytab as oryx/127.8.203.66@KRBTEST.COM (short username oryx)
I20260811 02:58:07.388700 26476 webserver.cc:460] Webserver started at http://127.8.203.66:40293/ using document root <none> and password file <none>
I20260811 02:58:07.389248 26476 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:07.389353 26476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:07.393169 26476 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 02:58:07.395254 26518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.396391 26476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:07.396497 26476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "aa076a47dc76443dbad266657ac5a8a2"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "d0d81551f2088a20d4ef8c79c8cab718"
server_key_iv: "fbcbc1b9571bf80bfcf8762baeb544a4"
server_key_version: "encryptionkey@0"
I20260811 02:58:07.397007 26476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:07.424497 26476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:07.427249 26476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:07.427426 26476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:07.428001 26476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:07.428807 26476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:07.428861 26476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.428917 26476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:07.428943 26476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.437248 26476 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:46597
I20260811 02:58:07.437325 26744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:46597 every 8 connection(s)
I20260811 02:58:07.438256 26476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:58:07.448055 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.439907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:59629 (local address 127.8.203.126:45705)
0811 02:58:07.440047 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.440051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.440839 (+ 788us) server_negotiation.cc:407] Connection header received
0811 02:58:07.442104 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.442107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.442158 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.442227 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.443290 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.443799 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.444818 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.445018 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.445082 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.447955 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:59629: BlockingRecv error: recv got EOF from 127.8.203.66:59629 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:58:07.448356 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 26476
I20260811 02:58:07.448629 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.440268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:59629)
0811 02:58:07.440688 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.440729 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.441915 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.442388 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.442395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.442719 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.443182 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.443195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.443881 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.443883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.444732 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.444741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.445120 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.445123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.445251 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.445824 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.445840 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.447830 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":116,"threads_started":1}
I20260811 02:58:07.448773 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:39117
--local_ip_for_outbound_sockets=127.8.203.67
--tserver_master_addrs=127.8.203.126:45705
--webserver_port=41997
--webserver_interface=127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:07.449208 26747 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:45705 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.203.126:45705: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:07.454320 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.449625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:45311)
0811 02:58:07.449774 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.449786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.449857 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.450046 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.450047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.450162 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.450241 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.450245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.450981 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.450984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.451728 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.451740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.452136 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.452139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.452207 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.452624 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.452635 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.454237 (+ 1602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 02:58:07.454511 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.449736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:45311 (local address 127.8.203.126:45705)
0811 02:58:07.449849 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.449853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.449866 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:58:07.449923 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.449925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.449965 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.450032 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.450328 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.450884 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.451830 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.452030 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.452110 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.454397 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:45311: BlockingRecv error: recv got EOF from 127.8.203.66:45311 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:07.459473 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.455012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:60889)
0811 02:58:07.455152 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.455164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.455223 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.455427 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.455429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.455542 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.455604 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.455608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.456507 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.456509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.457005 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.457011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.457532 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.457535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.457587 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.457966 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.457977 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.459395 (+ 1418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 02:58:07.459643 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.455115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:60889 (local address 127.8.203.126:45705)
0811 02:58:07.455228 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.455232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.455248 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:58:07.455283 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.455286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.455328 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.455424 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.455694 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.456421 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.457101 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.457437 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.457542 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.459538 (+ 1996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:60889: BlockingRecv error: recv got EOF from 127.8.203.66:60889 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 02:58:07.459748 26747 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:58:07.559058 26751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:07.559283 26751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:07.559340 26751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:07.563226 26751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:07.563275 26751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:07.563354 26751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:58:07.568689 26751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/krb5kdc/oryx_127.8.203.67.keytab
--principal=oryx/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:39117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=41997
--tserver_master_addrs=127.8.203.126:45705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:07.569695 26751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:07.570645 26751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:07.580058 26761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:07.580078 26760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:07.580078 26768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:07.580267 26751 server_base.cc:1034] running on GCE node
I20260811 02:58:07.580700 26751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:07.581261 26751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:07.582439 26751 hybrid_clock.cc:648] HybridClock initialized: now 1786417087582412 us; error 43 us; skew 500 ppm
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417087, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:07.584894 26751 init.cc:377] Logged in from keytab as oryx/127.8.203.67@KRBTEST.COM (short username oryx)
I20260811 02:58:07.585740 26751 webserver.cc:460] Webserver started at http://127.8.203.67:41997/ using document root <none> and password file <none>
I20260811 02:58:07.586239 26751 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:07.586323 26751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:07.589993 26751 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:58:07.591948 26782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.593104 26751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:07.593207 26751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "f162607cc1e345f5b5767cad175b8a5a"
format_stamp: "Formatted at 2026-08-11 02:58:06 on dist-test-slave-gjw7"
server_key: "a306018602ecf03fbe299fdb299e3b68"
server_key_iv: "fc5c5cf2300ae27c4ecf4838ca3de9b3"
server_key_version: "encryptionkey@0"
I20260811 02:58:07.593770 26751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:07.616632 26751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:07.619383 26751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:07.619565 26751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:07.620139 26751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:07.620985 26751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:07.621037 26751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.621095 26751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:07.621125 26751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:07.630290 26751 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:39117
I20260811 02:58:07.630314 27010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:39117 every 8 connection(s)
I20260811 02:58:07.631283 26751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:58:07.635656 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 26751
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:58:07.642944 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.632992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:53511 (local address 127.8.203.126:45705)
0811 02:58:07.633124 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.633128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.633952 (+ 824us) server_negotiation.cc:407] Connection header received
0811 02:58:07.635224 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.635228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.635285 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.635360 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.636780 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.637539 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.638994 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.639372 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.639477 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.642845 (+ 3368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:53511: BlockingRecv error: recv got EOF from 127.8.203.67:53511 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:58:07.643672 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.633378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:53511)
0811 02:58:07.633808 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.633850 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.635019 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.635551 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.635558 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.635942 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.636591 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.636609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.637658 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.637661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.638887 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.638897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.639498 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.639501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.639632 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.640192 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.640208 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.642744 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":113,"threads_started":1}
W20260811 02:58:07.644088 27011 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:45705 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.203.126:45705: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
I20260811 02:58:07.645553 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.636559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.8.203.126:45705)
0811 02:58:07.636792 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.636796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.636807 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:58:07.636841 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.636843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.636883 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.636949 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.637805 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.638590 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.639703 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.639998 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.640094 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.641639 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:07.641645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:07.641658 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:07.641684 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:07.643447 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:07.643862 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:07.643868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:07.643883 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:07.644040 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:07.644344 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:07.644347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:07.644348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:07.644781 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:07.644978 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:07.645187 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 02:58:07.645440 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:07.648988 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.644493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:44541)
0811 02:58:07.644591 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.644604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.644672 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.644861 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.644864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.645010 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.645108 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.645115 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.645922 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.645926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.646492 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.646501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.646909 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.646913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.646995 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.647362 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.647373 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.648906 (+ 1533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 02:58:07.649122 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.644589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:44541 (local address 127.8.203.126:45705)
0811 02:58:07.644693 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.644696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.644707 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:58:07.644735 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.644737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.644769 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.644825 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.645235 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.645796 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.646600 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.646811 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.646890 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.649023 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:44541: BlockingRecv error: recv got EOF from 127.8.203.67:44541 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:07 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:07.653221 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:07.649432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:40811)
0811 02:58:07.649574 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:07.649586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:07.649649 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:07.649820 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:07.649822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:07.649933 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:07.649995 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.649999 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.650573 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.650575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.651068 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:07.651074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.651391 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.651393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:07.651442 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.651754 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:07.651763 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:07.653126 (+ 1363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 02:58:07.653348 26471 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:07.649510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:40811 (local address 127.8.203.126:45705)
0811 02:58:07.649639 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:07.649642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:07.649653 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:58:07.649714 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:07.649716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:07.649745 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:07.649801 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:07.650066 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.650504 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.651143 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:07.651325 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:07.651384 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:07.653230 (+ 1846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:40811: BlockingRecv error: recv got EOF from 127.8.203.67:40811 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 02:58:07.653460 27011 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 02:58:08 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:08 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:08.274345 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:08.265585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:60553)
0811 02:58:08.265858 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:08.265881 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:08.265971 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:08.266173 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:08.266175 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:08.266350 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:08.266445 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:08.266450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.267121 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.267123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:08.267686 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:08.267693 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.268046 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.268048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:08.268105 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:08.272258 (+ 4153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:08.272269 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:08.274167 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
W20260811 02:58:08.274487 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:08.265632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:60553 (local address 127.8.203.126:45705)
0811 02:58:08.265908 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:08.265911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:08.265924 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:58:08.266043 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:08.266045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:08.266089 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:08.266159 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:08.266523 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.267045 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.267766 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.267974 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.268036 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:08.274352 (+ 6316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:60553: BlockingRecv error: recv got EOF from 127.8.203.65:60553 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":104,"threads_started":1}
Aug 11 02:58:08 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:08 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:08.465015 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:08.460274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:59403)
0811 02:58:08.460520 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:08.460534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:08.460627 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:08.460826 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:08.460828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:08.461000 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:08.461094 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:08.461099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.461770 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.461773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:08.462330 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:08.462348 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.462694 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.462696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:08.462749 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:08.463229 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:08.463241 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:08.464898 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
W20260811 02:58:08.465137 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:08.460358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:59403 (local address 127.8.203.126:45705)
0811 02:58:08.460492 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:08.460496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:08.460590 (+ 94us) server_negotiation.cc:407] Connection header received
0811 02:58:08.460694 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:08.460697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:08.460737 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:08.460800 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:08.461173 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.461696 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.462421 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.462622 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.462684 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:08.465034 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:59403: BlockingRecv error: recv got EOF from 127.8.203.66:59403 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 02:58:08 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:08 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:08.659247 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:08.654009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:50055)
0811 02:58:08.654275 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:08.654289 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:08.654391 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:08.654632 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:08.654634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:08.654831 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:08.654958 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:08.654964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.655724 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.655728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:08.656524 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:08.656534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.656963 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.656966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:08.657039 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:08.657516 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:08.657526 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:08.659138 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
W20260811 02:58:08.659401 27024 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:08.654097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:50055 (local address 127.8.203.126:45705)
0811 02:58:08.654239 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:08.654244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:08.654360 (+ 116us) server_negotiation.cc:407] Connection header received
0811 02:58:08.654478 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:08.654481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:08.654529 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:08.654593 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:08.655072 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.655608 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.656621 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:08.656868 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:08.656945 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:08.659278 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:50055: BlockingRecv error: recv got EOF from 127.8.203.67:50055 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 02:58:09 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:09 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:09.279978 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:09.275092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:37607)
0811 02:58:09.275349 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:09.275363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:09.275451 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:09.275660 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:09.275662 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:09.275838 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:09.275957 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:09.275963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.276640 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.276642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:09.277192 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:09.277199 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.277587 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.277590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:09.277654 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:09.278102 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:09.278114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:09.279833 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
W20260811 02:58:09.280104 27031 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:09.275168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:37607 (local address 127.8.203.126:45705)
0811 02:58:09.275429 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:09.275432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:09.275445 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:58:09.275522 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:09.275524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:09.275565 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:09.275637 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:09.276041 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.276567 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.277267 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.277500 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.277563 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:09.279973 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:37607: BlockingRecv error: recv got EOF from 127.8.203.65:37607 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
Aug 11 02:58:09 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:09 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:09.470960 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:09.465701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:34071)
0811 02:58:09.466030 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:09.466043 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:09.466148 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:09.466438 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:09.466441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:09.466646 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:09.466745 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:09.466751 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.467491 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.467494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:09.468047 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:09.468054 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.468476 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.468480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:09.468563 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:09.469028 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:09.469038 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:09.470775 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":93,"threads_started":1}
W20260811 02:58:09.471069 27031 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:09.465767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:34071 (local address 127.8.203.126:45705)
0811 02:58:09.465944 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:09.465949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:09.466108 (+ 159us) server_negotiation.cc:407] Connection header received
0811 02:58:09.466238 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:09.466241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:09.466296 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:09.466400 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:09.466858 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.467381 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.468158 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.468377 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.468461 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:09.470946 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:34071: BlockingRecv error: recv got EOF from 127.8.203.66:34071 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 02:58:09 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:09 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:09.664974 27038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:09.659999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:52673)
0811 02:58:09.660286 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:09.660300 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:09.660400 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:09.660598 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:09.660600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:09.660787 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:09.660934 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:09.660940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.661631 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.661634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:09.662182 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:09.662188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.662540 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.662543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:09.662596 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:09.663085 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:09.663097 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:09.664859 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":85,"threads_started":1}
W20260811 02:58:09.665103 27031 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:09.660121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:52673 (local address 127.8.203.126:45705)
0811 02:58:09.660268 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:09.660273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:09.660354 (+ 81us) server_negotiation.cc:407] Connection header received
0811 02:58:09.660463 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:09.660465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:09.660509 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:09.660575 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:09.661024 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.661562 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.662261 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:09.662467 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:09.662531 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:09.664989 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:52673: BlockingRecv error: recv got EOF from 127.8.203.67:52673 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 02:58:10 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:10 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:10.285622 27039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:10.280721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:36531)
0811 02:58:10.280993 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:10.281008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:10.281096 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:10.281348 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:10.281350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:10.281530 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:10.281631 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:10.281636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.282315 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.282317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:10.282859 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:10.282865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.283214 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.283216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:10.283270 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:10.283758 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:10.283770 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:10.285503 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
W20260811 02:58:10.285765 27040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:10.280803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:36531 (local address 127.8.203.126:45705)
0811 02:58:10.281031 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:10.281034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:10.281050 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:58:10.281170 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:10.281173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:10.281215 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:10.281284 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:10.281716 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.282241 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.282934 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.283138 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.283205 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:10.285629 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:36531: BlockingRecv error: recv got EOF from 127.8.203.65:36531 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
Aug 11 02:58:10 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:10 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:10.476552 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:10.471723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:34333)
0811 02:58:10.471971 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:10.471984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:10.472088 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:10.472292 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:10.472294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:10.472479 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:10.472577 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:10.472582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.473238 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.473241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:10.473836 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:10.473845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.474196 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.474199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:10.474253 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:10.474741 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:10.474754 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:10.476436 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
W20260811 02:58:10.476675 27040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:10.471793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:34333 (local address 127.8.203.126:45705)
0811 02:58:10.471951 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:10.471955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:10.472048 (+ 93us) server_negotiation.cc:407] Connection header received
0811 02:58:10.472156 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:10.472158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:10.472202 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:10.472264 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:10.472656 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.473166 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.473921 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.474123 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.474187 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:10.476568 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:34333: BlockingRecv error: recv got EOF from 127.8.203.66:34333 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 02:58:10 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:10 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:10.670784 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:10.665686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:39623)
0811 02:58:10.665958 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:10.665971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:10.666079 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:10.666316 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:10.666319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:10.666525 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:10.666624 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:10.666630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.667351 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.667355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:10.667927 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:10.667935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.668358 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.668362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:10.668452 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:10.668908 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:10.668919 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:10.670659 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
W20260811 02:58:10.670915 27040 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:10.665756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:39623 (local address 127.8.203.126:45705)
0811 02:58:10.665913 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:10.665917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:10.666031 (+ 114us) server_negotiation.cc:407] Connection header received
0811 02:58:10.666161 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:10.666165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:10.666216 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:10.666279 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:10.666732 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.667254 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.668037 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:10.668260 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:10.668344 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:10.670790 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:39623: BlockingRecv error: recv got EOF from 127.8.203.67:39623 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 02:58:11 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:11 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:11.291400 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:11.286336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:52643)
0811 02:58:11.286599 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:11.286612 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:11.286702 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:11.286927 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:11.286930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:11.287108 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:11.287208 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:11.287213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.287908 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.287910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:11.288454 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:11.288460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.288816 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.288818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:11.288901 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:11.289368 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:11.289378 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:11.291279 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
W20260811 02:58:11.291538 27047 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:11.286412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:52643 (local address 127.8.203.126:45705)
0811 02:58:11.286653 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:11.286656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:11.286670 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:58:11.286777 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:11.286780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:11.286822 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:11.286903 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:11.287291 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.287819 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.288529 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.288738 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.288803 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:11.291418 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:52643: BlockingRecv error: recv got EOF from 127.8.203.65:52643 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
Aug 11 02:58:11 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:11 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:11.482748 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:11.477284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:35583)
0811 02:58:11.477590 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:11.477603 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:11.477710 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:11.477956 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:11.477959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:11.478161 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:11.478258 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:11.478263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.479006 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.479010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:11.479561 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:11.479570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.480001 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.480004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:11.480086 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:11.480581 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:11.480593 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:11.482600 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
W20260811 02:58:11.482841 27047 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:11.477369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:35583 (local address 127.8.203.126:45705)
0811 02:58:11.477502 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:11.477507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:11.477670 (+ 163us) server_negotiation.cc:407] Connection header received
0811 02:58:11.477798 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:11.477801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:11.477851 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:11.477918 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:11.478378 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.478903 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.479675 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.479902 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.479987 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:11.482738 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:35583: BlockingRecv error: recv got EOF from 127.8.203.66:35583 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 02:58:11 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:11 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:11.676630 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:11.671472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:35577)
0811 02:58:11.671738 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:11.671753 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:11.671881 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:11.672126 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:11.672129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:11.672327 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:11.672428 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:11.672434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.673160 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.673164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:11.673755 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:11.673764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.674186 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.674189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:11.674272 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:11.674754 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:11.674767 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:11.676482 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
W20260811 02:58:11.676823 27047 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:11.671538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:35577 (local address 127.8.203.126:45705)
0811 02:58:11.671660 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:11.671663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:11.671830 (+ 167us) server_negotiation.cc:407] Connection header received
0811 02:58:11.671969 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:11.671972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:11.672020 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:11.672081 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:11.672543 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.673055 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.673867 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:11.674085 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:11.674171 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:11.676620 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:35577: BlockingRecv error: recv got EOF from 127.8.203.67:35577 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 02:58:12 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:12 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:12.297251 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:12.292132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:42561)
0811 02:58:12.292398 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:12.292412 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:12.292504 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:12.292713 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:12.292715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:12.292909 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:12.293008 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:12.293013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.293768 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.293771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:12.294342 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:12.294351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.294737 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.294739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:12.294832 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:12.295327 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:12.295342 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:12.297095 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
W20260811 02:58:12.297416 27055 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:12.292196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:42561 (local address 127.8.203.126:45705)
0811 02:58:12.292426 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:12.292430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:12.292461 (+ 31us) server_negotiation.cc:407] Connection header received
0811 02:58:12.292574 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:12.292576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:12.292618 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:12.292690 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:12.293093 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.293670 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.294437 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.294657 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.294726 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:12.297262 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:42561: BlockingRecv error: recv got EOF from 127.8.203.65:42561 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":69,"threads_started":1}
Aug 11 02:58:12 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:12 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:12.488847 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:12.483523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:60165)
0811 02:58:12.483806 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:12.483821 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:12.483930 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:12.484168 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:12.484170 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:12.484393 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:12.484488 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:12.484495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.485233 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.485237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:12.485856 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:12.485866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.486290 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.486293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:12.486388 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:12.486870 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:12.486885 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:12.488703 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
W20260811 02:58:12.488976 27055 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:12.483677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:60165 (local address 127.8.203.126:45705)
0811 02:58:12.483808 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:12.483812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:12.483876 (+ 64us) server_negotiation.cc:407] Connection header received
0811 02:58:12.484021 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:12.484024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:12.484067 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:12.484133 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:12.484605 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.485129 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.485975 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.486190 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.486275 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:12.488860 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:60165: BlockingRecv error: recv got EOF from 127.8.203.66:60165 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 02:58:12 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:12 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:12.682734 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:12.677448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:44795)
0811 02:58:12.677731 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:12.677744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:12.677837 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:12.678046 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:12.678048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:12.678233 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:12.678326 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:12.678331 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.679234 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.679239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:12.679966 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:12.679977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.680398 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.680400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:12.680457 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:12.680908 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:12.680919 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:12.682620 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
W20260811 02:58:12.682858 27055 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:12.677513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:44795 (local address 127.8.203.126:45705)
0811 02:58:12.677643 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:12.677646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:12.677804 (+ 158us) server_negotiation.cc:407] Connection header received
0811 02:58:12.677909 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:12.677911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:12.677955 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:12.678021 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:12.678410 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.679121 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.680090 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:12.680305 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:12.680385 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:12.682747 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:44795: BlockingRecv error: recv got EOF from 127.8.203.67:44795 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 02:58:13 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:13 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:13.307358 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:13.298115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:41095)
0811 02:58:13.298435 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:13.298452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:13.298555 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:13.298769 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:13.298771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:13.298962 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:13.299061 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:13.299066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.299764 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.299766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:13.300383 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:13.300393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.300807 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.300810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:13.300870 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:13.305089 (+ 4219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:13.305102 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:13.307203 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":87,"threads_started":1}
W20260811 02:58:13.307463 27062 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:13.298186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:41095 (local address 127.8.203.126:45705)
0811 02:58:13.298459 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:13.298463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:13.298523 (+ 60us) server_negotiation.cc:407] Connection header received
0811 02:58:13.298626 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:13.298628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:13.298672 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:13.298746 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:13.299147 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.299687 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.300490 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.300720 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.300799 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:13.307344 (+ 6545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:41095: BlockingRecv error: recv got EOF from 127.8.203.65:41095 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":78,"threads_started":1}
Aug 11 02:58:13 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:13 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:13.495003 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:13.489651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:55605)
0811 02:58:13.489930 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:13.489943 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:13.490051 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:13.490285 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:13.490288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:13.490534 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:13.490650 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:13.490657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.491421 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.491424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:13.491974 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:13.491981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.492431 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.492433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:13.492491 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:13.492980 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:13.492994 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:13.494889 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
W20260811 02:58:13.495136 27062 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:13.489724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:55605 (local address 127.8.203.126:45705)
0811 02:58:13.489877 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:13.489881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:13.490003 (+ 122us) server_negotiation.cc:407] Connection header received
0811 02:58:13.490133 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:13.490136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:13.490181 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:13.490268 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:13.490770 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.491322 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.492097 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.492338 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.492424 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:13.495017 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:55605: BlockingRecv error: recv got EOF from 127.8.203.66:55605 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 02:58:13 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:13 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:13.688473 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:13.683483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:36083)
0811 02:58:13.683747 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:13.683760 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:13.683859 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:13.684071 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:13.684073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:13.684260 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:13.684369 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:13.684375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.685036 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.685038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:13.685667 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:13.685678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.686040 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.686042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:13.686098 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:13.686578 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:13.686590 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:13.688353 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":72,"threads_started":1}
W20260811 02:58:13.688613 27062 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:13.683583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:36083 (local address 127.8.203.126:45705)
0811 02:58:13.683707 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:13.683711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:13.683820 (+ 109us) server_negotiation.cc:407] Connection header received
0811 02:58:13.683930 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:13.683932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:13.683980 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:13.684044 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:13.684450 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.684964 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.685754 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:13.685966 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:13.686030 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:13.688501 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:36083: BlockingRecv error: recv got EOF from 127.8.203.67:36083 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 02:58:14 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:14 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:14.313241 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:14.308202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:39599)
0811 02:58:14.308557 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:14.308570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:14.308675 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:14.308888 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:14.308890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:14.309072 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:14.309170 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:14.309175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.309885 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.309887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:14.310451 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:14.310459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.310813 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.310816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:14.310872 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:14.311367 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:14.311380 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:14.313126 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":99,"threads_started":1}
W20260811 02:58:14.313436 27068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:14.308284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:39599 (local address 127.8.203.126:45705)
0811 02:58:14.308557 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:14.308561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:14.308635 (+ 74us) server_negotiation.cc:407] Connection header received
0811 02:58:14.308746 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:14.308748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:14.308791 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:14.308860 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:14.309276 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.309810 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.310530 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.310734 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.310802 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:14.313265 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:39599: BlockingRecv error: recv got EOF from 127.8.203.65:39599 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":69,"threads_started":1}
Aug 11 02:58:14 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:14 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:14.501212 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:14.495738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:50645)
0811 02:58:14.496178 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:14.496194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:14.496300 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:14.496579 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:14.496583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:14.496831 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:14.496942 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:14.496949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.497789 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.497792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:14.498386 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:14.498396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.498833 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.498836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:14.498896 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:14.499327 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:14.499338 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:14.501082 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":107,"threads_started":1}
W20260811 02:58:14.501394 27068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:14.495812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:50645 (local address 127.8.203.126:45705)
0811 02:58:14.495979 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:14.495985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:14.496270 (+ 285us) server_negotiation.cc:407] Connection header received
0811 02:58:14.496394 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:14.496396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:14.496450 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:14.496527 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:14.497061 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.497709 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.498489 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.498754 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.498829 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:14.501223 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:50645: BlockingRecv error: recv got EOF from 127.8.203.66:50645 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 02:58:14 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:14 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:14.694105 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:14.689146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:50359)
0811 02:58:14.689460 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:14.689473 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:14.689568 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:14.689769 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:14.689771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:14.689965 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:14.690060 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:14.690065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.690729 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.690732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:14.691280 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:14.691286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.691637 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.691639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:14.691694 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:14.692136 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:14.692146 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:14.693989 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":75,"threads_started":1}
W20260811 02:58:14.694231 27068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:14.689219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:50359 (local address 127.8.203.126:45705)
0811 02:58:14.689439 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:14.689443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:14.689534 (+ 91us) server_negotiation.cc:407] Connection header received
0811 02:58:14.689636 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:14.689638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:14.689684 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:14.689746 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:14.690138 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.690654 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.691363 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:14.691564 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:14.691628 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:14.694113 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:50359: BlockingRecv error: recv got EOF from 127.8.203.67:50359 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 11 02:58:15 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:15 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:15.318914 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:15.313971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:50883)
0811 02:58:15.314227 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:15.314241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:15.314328 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:15.314551 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:15.314554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:15.314734 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:15.314846 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:15.314852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.315601 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.315604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:15.316176 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:15.316184 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.316565 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.316568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:15.316634 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:15.317087 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:15.317098 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:15.318788 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":73,"threads_started":1}
W20260811 02:58:15.319046 27079 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:15.314092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:50883 (local address 127.8.203.126:45705)
0811 02:58:15.314302 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:15.314305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:15.314317 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:58:15.314404 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:15.314406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:15.314457 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:15.314527 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:15.314927 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.315517 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.316259 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.316476 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.316564 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:15.318925 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:50883: BlockingRecv error: recv got EOF from 127.8.203.65:50883 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":64,"threads_started":1}
Aug 11 02:58:15 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:15 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:15.506886 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:15.501893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:53053)
0811 02:58:15.502150 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:15.502164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:15.502264 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:15.502471 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:15.502473 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:15.502650 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:15.502747 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:15.502752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.503551 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.503553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:15.504116 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:15.504124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.504483 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.504485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:15.504541 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:15.504982 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:15.504994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:15.506733 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
W20260811 02:58:15.506983 27079 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:15.501960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:53053 (local address 127.8.203.126:45705)
0811 02:58:15.502083 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:15.502086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:15.502221 (+ 135us) server_negotiation.cc:407] Connection header received
0811 02:58:15.502341 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:15.502343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:15.502388 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:15.502443 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:15.502826 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.503474 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.504199 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.504408 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.504474 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:15.506880 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:53053: BlockingRecv error: recv got EOF from 127.8.203.66:53053 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 02:58:15 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:15 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:15.700163 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:15.694843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:37849)
0811 02:58:15.695116 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:15.695128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:15.695239 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:15.695489 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:15.695492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:15.695720 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:15.695847 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:15.695854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.696604 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.696608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:15.697170 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:15.697178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.697685 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.697689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:15.697770 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:15.698201 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:15.698211 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:15.700015 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":86,"threads_started":1}
W20260811 02:58:15.700253 27079 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:15.694897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:37849 (local address 127.8.203.126:45705)
0811 02:58:15.695041 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:15.695046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:15.695195 (+ 149us) server_negotiation.cc:407] Connection header received
0811 02:58:15.695327 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:15.695329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:15.695389 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:15.695454 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:15.695965 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.696501 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.697281 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:15.697583 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:15.697677 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:15.700150 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:37849: BlockingRecv error: recv got EOF from 127.8.203.67:37849 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 02:58:16 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:16 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:16.324685 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:16.319641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:35359)
0811 02:58:16.319921 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:16.319934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:16.320028 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:16.320236 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:16.320238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:16.320417 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:16.320514 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:16.320519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.321196 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.321198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:16.321783 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:16.321792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.322175 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.322177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:16.322234 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:16.322706 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:16.322718 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:16.324548 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
W20260811 02:58:16.324815 27085 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:16.319716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:35359 (local address 127.8.203.126:45705)
0811 02:58:16.319940 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:16.319943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:16.319990 (+ 47us) server_negotiation.cc:407] Connection header received
0811 02:58:16.320097 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:16.320099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:16.320141 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:16.320213 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:16.320594 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.321119 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.321874 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.322094 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.322168 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:16.324674 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:35359: BlockingRecv error: recv got EOF from 127.8.203.65:35359 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":61,"threads_started":1}
Aug 11 02:58:16 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:16 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:16.512365 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:16.507610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:34429)
0811 02:58:16.507858 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:16.507871 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:16.507968 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:16.508167 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:16.508169 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:16.508351 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:16.508460 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:16.508465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.509115 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.509117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:16.509714 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:16.509723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.510069 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.510072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:16.510125 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:16.510560 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:16.510571 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:16.512227 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
W20260811 02:58:16.512459 27085 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:16.507651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:34429 (local address 127.8.203.126:45705)
0811 02:58:16.507780 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:16.507783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:16.507928 (+ 145us) server_negotiation.cc:407] Connection header received
0811 02:58:16.508035 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:16.508037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:16.508080 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:16.508142 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:16.508537 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.509040 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.509797 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.509997 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.510061 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:16.512364 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:34429: BlockingRecv error: recv got EOF from 127.8.203.66:34429 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 02:58:16 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:16 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:16.706100 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:16.700878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:46775)
0811 02:58:16.701180 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:16.701193 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:16.701319 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:16.701571 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:16.701574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:16.701782 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:16.701878 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:16.701884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.702620 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.702623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:16.703170 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:16.703177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.703611 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.703615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:16.703700 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:16.704181 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:16.704193 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:16.705988 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":128,"threads_started":1}
W20260811 02:58:16.706230 27085 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:16.700977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:46775 (local address 127.8.203.126:45705)
0811 02:58:16.701118 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:16.701122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:16.701256 (+ 134us) server_negotiation.cc:407] Connection header received
0811 02:58:16.701416 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:16.701419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:16.701470 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:16.701534 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:16.701992 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.702515 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.703281 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:16.703509 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:16.703595 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:16.706121 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:46775: BlockingRecv error: recv got EOF from 127.8.203.67:46775 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 02:58:17 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:17 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:17.330296 27092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:17.325439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:44193)
0811 02:58:17.325693 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:17.325707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:17.325795 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:17.326006 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:17.326008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:17.326192 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:17.326290 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:17.326295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.326976 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.326978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:17.327532 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:17.327539 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.327891 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.327893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:17.327949 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:17.328426 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:17.328438 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:17.330175 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":70,"threads_started":1}
W20260811 02:58:17.330446 27093 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:17.325496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:44193 (local address 127.8.203.126:45705)
0811 02:58:17.325723 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:17.325726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:17.325753 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:58:17.325866 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:17.325868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:17.325912 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:17.325983 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:17.326380 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.326900 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.327610 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.327819 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.327884 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:17.330299 (+ 2415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:44193: BlockingRecv error: recv got EOF from 127.8.203.65:44193 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
Aug 11 02:58:17 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:17 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:17.518091 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:17.513075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:54183)
0811 02:58:17.513378 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:17.513393 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:17.513486 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:17.513674 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:17.513676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:17.513852 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:17.513947 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:17.513953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.514612 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.514614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:17.515239 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:17.515251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.515644 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.515647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:17.515702 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:17.516241 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:17.516253 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:17.517974 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
W20260811 02:58:17.518206 27093 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:17.513144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:54183 (local address 127.8.203.126:45705)
0811 02:58:17.513279 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:17.513282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:17.513448 (+ 166us) server_negotiation.cc:407] Connection header received
0811 02:58:17.513548 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:17.513550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:17.513590 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:17.513651 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:17.514026 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.514539 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.515337 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.515563 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.515636 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:17.518103 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:54183: BlockingRecv error: recv got EOF from 127.8.203.66:54183 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 02:58:17 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:17 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:17.711964 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:17.706805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:53165)
0811 02:58:17.707092 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:17.707108 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:17.707215 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:17.707471 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:17.707474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:17.707678 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:17.707773 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:17.707780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.708515 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.708519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:17.709082 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:17.709090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.709584 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.709588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:17.709672 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:17.710115 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:17.710126 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:17.711817 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
W20260811 02:58:17.712065 27093 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:17.706964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:53165 (local address 127.8.203.126:45705)
0811 02:58:17.707092 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:17.707096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:17.707163 (+ 67us) server_negotiation.cc:407] Connection header received
0811 02:58:17.707303 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:17.707306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:17.707366 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:17.707431 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:17.707890 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.708413 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.709194 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:17.709478 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:17.709568 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:17.711963 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:53165: BlockingRecv error: recv got EOF from 127.8.203.67:53165 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 02:58:18 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:18 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:18.340909 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:18.331020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:42223)
0811 02:58:18.331321 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:18.331337 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:18.331454 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:18.331727 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:18.331730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:18.331956 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:18.332081 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:18.332087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.332857 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.332860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:18.333450 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:18.333459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.333836 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.333838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:18.333900 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:18.339020 (+ 5120us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:18.339031 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:18.340756 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":89,"threads_started":1}
W20260811 02:58:18.341035 27102 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:18.331115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:42223 (local address 127.8.203.126:45705)
0811 02:58:18.331336 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:18.331340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:18.331411 (+ 71us) server_negotiation.cc:407] Connection header received
0811 02:58:18.331539 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:18.331542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:18.331599 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:18.331691 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:18.332206 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.332762 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.333536 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.333755 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.333825 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:18.340917 (+ 7092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:42223: BlockingRecv error: recv got EOF from 127.8.203.65:42223 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":61,"threads_started":1}
Aug 11 02:58:18 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:18 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:18.523942 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:18.518785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:38801)
0811 02:58:18.519077 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:18.519089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:18.519197 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:18.519437 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:18.519440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:18.519642 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:18.519740 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:18.519746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.520490 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.520495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:18.521044 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:18.521052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.521541 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.521545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:18.521628 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:18.522114 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:18.522126 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:18.523815 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":107,"threads_started":1}
W20260811 02:58:18.524057 27102 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:18.518847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:38801 (local address 127.8.203.126:45705)
0811 02:58:18.518994 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:18.518999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:18.519155 (+ 156us) server_negotiation.cc:407] Connection header received
0811 02:58:18.519281 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:18.519284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:18.519332 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:18.519401 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:18.519864 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.520383 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.521158 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.521427 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.521512 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:18.523955 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:38801: BlockingRecv error: recv got EOF from 127.8.203.66:38801 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 02:58:18 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:18 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:18.717622 27105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:18.712667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:57299)
0811 02:58:18.712921 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:18.712935 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:18.713032 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:18.713239 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:18.713241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:18.713448 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:18.713547 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:18.713552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.714221 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.714223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:18.714758 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:18.714764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.715167 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.715169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:18.715231 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:18.715723 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:18.715734 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:18.717486 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
W20260811 02:58:18.717715 27102 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:18.712731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:57299 (local address 127.8.203.126:45705)
0811 02:58:18.712895 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:18.712899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:18.712994 (+ 95us) server_negotiation.cc:407] Connection header received
0811 02:58:18.713102 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:18.713104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:18.713151 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:18.713212 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:18.713626 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.714140 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.714868 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:18.715085 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:18.715154 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:18.717609 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:57299: BlockingRecv error: recv got EOF from 127.8.203.67:57299 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 02:58:19 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:19 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:19.346426 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:19.341667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:44503)
0811 02:58:19.341930 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:19.341943 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:19.342043 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:19.342240 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:19.342242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:19.342425 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:19.342528 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:19.342534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.343202 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.343204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:19.343752 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:19.343759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.344115 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.344117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:19.344174 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:19.344606 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:19.344616 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:19.346301 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
W20260811 02:58:19.346570 27108 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:19.341705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:44503 (local address 127.8.203.126:45705)
0811 02:58:19.341944 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:19.341947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:19.342004 (+ 57us) server_negotiation.cc:407] Connection header received
0811 02:58:19.342110 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:19.342112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:19.342155 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:19.342227 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:19.342609 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.343127 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.343829 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.344041 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.344107 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:19.346438 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:44503: BlockingRecv error: recv got EOF from 127.8.203.65:44503 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
Aug 11 02:58:19 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:19 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:19.529474 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:19.524557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:46879)
0811 02:58:19.524828 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:19.524848 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:19.524949 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:19.525153 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:19.525155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:19.525364 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:19.525465 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:19.525470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.526145 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.526147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:19.526688 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:19.526694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.527045 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.527047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:19.527100 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:19.527602 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:19.527615 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:19.529356 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
W20260811 02:58:19.529541 27108 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:19.524623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:46879 (local address 127.8.203.126:45705)
0811 02:58:19.524792 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:19.524797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:19.524912 (+ 115us) server_negotiation.cc:407] Connection header received
0811 02:58:19.525018 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:19.525020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:19.525065 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:19.525128 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:19.525545 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.526066 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.526762 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.526972 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.527037 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:19.529441 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:46879: BlockingRecv error: recv got EOF from 127.8.203.66:46879 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 02:58:19 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:19 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:19.723680 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:19.718300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:36759)
0811 02:58:19.718561 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:19.718575 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:19.718693 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:19.718977 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:19.718980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:19.719195 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:19.719324 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:19.719333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.720252 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.720255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:19.720802 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:19.720810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.721189 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.721191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:19.721247 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:19.721782 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:19.721797 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:19.723555 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":76,"threads_started":1}
W20260811 02:58:19.723791 27108 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:19.718381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:36759 (local address 127.8.203.126:45705)
0811 02:58:19.718523 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:19.718526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:19.718636 (+ 110us) server_negotiation.cc:407] Connection header received
0811 02:58:19.718779 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:19.718782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:19.718842 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:19.718927 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:19.719451 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.720157 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.720906 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:19.721115 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:19.721181 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:19.723688 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:36759: BlockingRecv error: recv got EOF from 127.8.203.67:36759 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 02:58:20 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:20 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:20.352414 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:20.347138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:48521)
0811 02:58:20.347457 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:20.347473 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:20.347587 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:20.347870 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:20.347874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:20.348080 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:20.348178 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:20.348185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.348940 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.348944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:20.349595 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:20.349605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.350045 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.350049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:20.350133 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:20.350580 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:20.350590 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:20.352236 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
W20260811 02:58:20.352522 27118 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:20.347189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:48521 (local address 127.8.203.126:45705)
0811 02:58:20.347470 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:20.347474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:20.347532 (+ 58us) server_negotiation.cc:407] Connection header received
0811 02:58:20.347674 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:20.347678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:20.347734 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:20.347825 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:20.348298 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.348832 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.349708 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.349941 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.350030 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:20.352402 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:48521: BlockingRecv error: recv got EOF from 127.8.203.65:48521 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":65,"threads_started":1}
Aug 11 02:58:20 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:20 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:20.535259 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:20.530201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:41647)
0811 02:58:20.530464 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:20.530480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:20.530595 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:20.530829 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:20.530832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:20.531044 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:20.531142 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:20.531148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.531897 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.531901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:20.532464 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:20.532472 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.532910 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.532914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:20.532993 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:20.533450 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:20.533462 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:20.535131 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
W20260811 02:58:20.535367 27118 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:20.530356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:41647 (local address 127.8.203.126:45705)
0811 02:58:20.530493 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:20.530497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:20.530535 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:58:20.530680 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:20.530683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:20.530732 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:20.530793 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:20.531256 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.531788 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.532579 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.532810 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.532897 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:20.535265 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:41647: BlockingRecv error: recv got EOF from 127.8.203.66:41647 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 02:58:20 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:20 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:20.729866 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:20.724432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:41257)
0811 02:58:20.724707 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:20.724721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:20.724829 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:20.725052 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:20.725054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:20.725271 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:20.725434 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:20.725442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.726150 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.726153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:20.726744 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:20.726754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.727127 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.727129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:20.727206 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:20.727677 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:20.727691 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:20.729744 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":86,"threads_started":1}
W20260811 02:58:20.729967 27118 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:20.724506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:41257 (local address 127.8.203.126:45705)
0811 02:58:20.724636 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:20.724639 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:20.724785 (+ 146us) server_negotiation.cc:407] Connection header received
0811 02:58:20.724904 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:20.724906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:20.724951 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:20.725014 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:20.725533 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.726049 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.726839 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:20.727046 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:20.727110 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:20.729867 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:41257: BlockingRecv error: recv got EOF from 127.8.203.67:41257 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 02:58:21 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:21 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:21.358424 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:21.353232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:39705)
0811 02:58:21.353523 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:21.353536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:21.353627 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:21.353870 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:21.353872 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:21.354057 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:21.354154 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:21.354159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.354827 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.354829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:21.355402 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:21.355410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.355771 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.355773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:21.355830 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:21.356275 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:21.356286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:21.358286 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":109,"threads_started":1}
W20260811 02:58:21.358522 27128 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:21.353383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:39705 (local address 127.8.203.126:45705)
0811 02:58:21.353683 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:21.353687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:21.353698 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:58:21.353732 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:21.353734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:21.353774 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:21.353843 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:21.354238 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.354753 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.355481 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.355697 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.355760 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:21.358411 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:39705: BlockingRecv error: recv got EOF from 127.8.203.65:39705 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":64,"threads_started":1}
Aug 11 02:58:21 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:21 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:21.540928 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:21.535951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:47861)
0811 02:58:21.536232 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:21.536245 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:21.536345 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:21.536608 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:21.536610 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:21.536848 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:21.536957 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:21.536964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.537722 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.537724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:21.538293 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:21.538301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.538677 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.538680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:21.538737 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:21.539181 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:21.539193 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:21.540796 (+ 1603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
W20260811 02:58:21.541021 27128 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:21.535995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:47861 (local address 127.8.203.126:45705)
0811 02:58:21.536130 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:21.536135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:21.536305 (+ 170us) server_negotiation.cc:407] Connection header received
0811 02:58:21.536435 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:21.536438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:21.536494 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:21.536570 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:21.537059 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.537643 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.538387 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.538600 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.538670 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:21.540925 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:47861: BlockingRecv error: recv got EOF from 127.8.203.66:47861 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 02:58:21 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:21 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.67@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:21.735607 27132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:21.730605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.67:57041)
0811 02:58:21.730876 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:21.730889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:21.730998 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:21.731240 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:21.731242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:21.731456 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:21.731552 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:21.731558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.732291 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.732295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:21.732851 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:21.732859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.733275 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.733279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:21.733403 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:21.733846 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:21.733857 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:21.735489 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":90,"threads_started":1}
W20260811 02:58:21.735733 27128 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:21.730688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:57041 (local address 127.8.203.126:45705)
0811 02:58:21.730819 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:21.730824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:21.730952 (+ 128us) server_negotiation.cc:407] Connection header received
0811 02:58:21.731093 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:21.731096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:21.731143 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:21.731205 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:21.731658 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.732188 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.732956 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:21.733178 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:21.733262 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:21.735623 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.67:57041: BlockingRecv error: recv got EOF from 127.8.203.67:57041 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 02:58:22 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:22 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.65@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:22.364676 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:22.359214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.65:40377)
0811 02:58:22.359494 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:22.359508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:22.359600 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:22.359806 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:22.359808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:22.359990 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:22.360087 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:22.360091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.360771 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.360774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:22.361362 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:22.361371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.361898 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.361900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:22.361962 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:22.362481 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:22.362492 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:22.364558 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
W20260811 02:58:22.364796 27135 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:22.359284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:40377 (local address 127.8.203.126:45705)
0811 02:58:22.359518 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:22.359521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:22.359569 (+ 48us) server_negotiation.cc:407] Connection header received
0811 02:58:22.359668 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:22.359670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:22.359713 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:22.359783 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:22.360166 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.360683 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.361450 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.361790 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.361897 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:22.364684 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.65:40377: BlockingRecv error: recv got EOF from 127.8.203.65:40377 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":56,"threads_started":1}
Aug 11 02:58:22 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:58:22 dist-test-slave-gjw7 krb5kdc[24939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.203.66@KRBTEST.COM for oryx/127.8.203.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:58:22.546941 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:22.541849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45705 (local address 127.8.203.66:58991)
0811 02:58:22.542144 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:22.542156 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:22.542258 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:22.542497 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:22.542500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:22.542704 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:22.542799 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:22.542805 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.543536 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.543540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:22.544092 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:22.544100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.544531 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.544535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:22.544619 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:22.545061 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:22.545072 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:22.546796 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.203.126:45705: Server oryx/127.8.203.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
W20260811 02:58:22.547045 27135 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:22.541908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:58991 (local address 127.8.203.126:45705)
0811 02:58:22.542064 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:22.542069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:22.542221 (+ 152us) server_negotiation.cc:407] Connection header received
0811 02:58:22.542344 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:22.542347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:22.542396 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:22.542459 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:22.542914 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.543437 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.544204 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.544433 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.544517 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:22.546946 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.203.66:58991: BlockingRecv error: recv got EOF from 127.8.203.66:58991 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:58:22.641175 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 26172
I20260811 02:58:22.659631 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 26476
I20260811 02:58:22.677348 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 26751
I20260811 02:58:22.688525 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 26018
2026-08-11T02:58:22Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18177 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T02:58:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:22Z Disabled control of system clock
I20260811 02:58:22.724087 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:37003
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:37003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 02:58:22.833973 27142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:22.834198 27142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:22.834229 27142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:22.838039 27142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:22.838083 27142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:22.838096 27142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:22.838110 27142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:22.838130 27142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:22.842643 27142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:37003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:37003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:22.843531 27142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:22.844744 27142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:22.852624 27157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:22.852775 27155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:22.852820 27142 server_base.cc:1034] running on GCE node
W20260811 02:58:22.852730 27162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:22.853479 27142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:22.854498 27142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:22.855654 27142 hybrid_clock.cc:648] HybridClock initialized: now 1786417102855633 us; error 30 us; skew 500 ppm
I20260811 02:58:22.857357 27142 webserver.cc:460] Webserver started at http://127.8.203.126:43609/ using document root <none> and password file <none>
I20260811 02:58:22.857884 27142 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:22.857949 27142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:22.858130 27142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:22.859985 27142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "c8baa2af633149f6813cd9adf4eeb6ef"
format_stamp: "Formatted at 2026-08-11 02:58:22 on dist-test-slave-gjw7"
server_key: "162031aadf949113b745f82a3e20eb32"
server_key_iv: "8cc10093ce560c7af5b360eb05334357"
server_key_version: "encryptionkey@0"
I20260811 02:58:22.860509 27142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "c8baa2af633149f6813cd9adf4eeb6ef"
format_stamp: "Formatted at 2026-08-11 02:58:22 on dist-test-slave-gjw7"
server_key: "162031aadf949113b745f82a3e20eb32"
server_key_iv: "8cc10093ce560c7af5b360eb05334357"
server_key_version: "encryptionkey@0"
I20260811 02:58:22.863870 27142 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 02:58:22.866106 27171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:22.867183 27142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:58:22.867293 27142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "c8baa2af633149f6813cd9adf4eeb6ef"
format_stamp: "Formatted at 2026-08-11 02:58:22 on dist-test-slave-gjw7"
server_key: "162031aadf949113b745f82a3e20eb32"
server_key_iv: "8cc10093ce560c7af5b360eb05334357"
server_key_version: "encryptionkey@0"
I20260811 02:58:22.867383 27142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:22.883477 27142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:22.884061 27142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:22.884219 27142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:22.890568 27142 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:37003
I20260811 02:58:22.890586 27267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:37003 every 8 connection(s)
I20260811 02:58:22.891405 27142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:22.894054 27268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:22.899147 27268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef: Bootstrap starting.
I20260811 02:58:22.899461 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 27142
I20260811 02:58:22.899561 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:58:22.899796 9005 external_mini_cluster.cc:1428] Setting key 3c0a1b80f5bebb399d6fd200140ac118
I20260811 02:58:22.901531 27268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:22.902385 27268 log.cc:824] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:22.904441 27268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef: No bootstrap required, opened a new log
I20260811 02:58:22.908413 27268 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37003 } }
I20260811 02:58:22.908664 27268 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:22.908718 27268 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8baa2af633149f6813cd9adf4eeb6ef, State: Initialized, Role: FOLLOWER
I20260811 02:58:22.909158 27268 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37003 } }
I20260811 02:58:22.909330 27268 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:22.909409 27268 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:22.909449 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:22.901013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.8.203.126:37003)
0811 02:58:22.901687 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:22.901711 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:22.901754 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:58:22.902510 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:22.902533 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:22.902979 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:22.903303 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:22.903699 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.905114 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.906002 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:22.906685 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:22.906841 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:22.907515 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:22.907523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:22.907538 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:22.908023 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:22.908092 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:22.908118 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:22.908210 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:22.908280 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 02:58:22.908526 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":100,"threads_started":1}
I20260811 02:58:22.909505 27268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:22.910256 27268 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37003 } }
I20260811 02:58:22.910604 27268 leader_election.cc:302] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8baa2af633149f6813cd9adf4eeb6ef; no voters:
I20260811 02:58:22.910902 27268 leader_election.cc:288] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:22.910986 27278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:22.911199 27278 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [term 1 LEADER]: Becoming Leader. State: Replica: c8baa2af633149f6813cd9adf4eeb6ef, State: Running, Role: LEADER
I20260811 02:58:22.911559 27278 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37003 } }
I20260811 02:58:22.912019 27268 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:22.913103 27283 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8baa2af633149f6813cd9adf4eeb6ef. Latest consensus state: current_term: 1 leader_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37003 } } }
I20260811 02:58:22.913228 27283 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:22.913385 27282 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8baa2af633149f6813cd9adf4eeb6ef" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 37003 } } }
I20260811 02:58:22.913575 27282 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef [sys.catalog]: This master's current role is: LEADER
W20260811 02:58:22.916460 27311 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:58:22.916525 27311 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:58:22.916604 27299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:22.917517 27299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:22.922794 27299 catalog_manager.cc:1347] Generated new cluster ID: 3b6aa001b56448ccae86b5746969395a
I20260811 02:58:22.922840 27299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:22.938067 27299 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:58:22.938853 27299 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:22.947475 27299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8baa2af633149f6813cd9adf4eeb6ef: Generated new TSK 0
I20260811 02:58:22.948025 27299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:58:22.955680 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:37003
--builtin_ntp_servers=127.8.203.84:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 02:58:23.066170 27315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:23.066402 27315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:23.066447 27315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:23.070341 27315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:23.070394 27315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:23.070474 27315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:58:23.075121 27315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:23.076076 27315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:23.077451 27315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:23.086534 27326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:23.086555 27331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:23.086711 27315 server_base.cc:1034] running on GCE node
W20260811 02:58:23.086555 27329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:23.087194 27315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:23.087718 27315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:23.088878 27315 hybrid_clock.cc:648] HybridClock initialized: now 1786417103088865 us; error 27 us; skew 500 ppm
I20260811 02:58:23.090613 27315 webserver.cc:460] Webserver started at http://127.8.203.65:44469/ using document root <none> and password file <none>
I20260811 02:58:23.091144 27315 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:23.091209 27315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:23.091418 27315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:23.093351 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "adb882d9297a492eba3d3c1bbb9fb205"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "6f717107fdcfd5e4c514f0e78dc2956b"
server_key_iv: "8bdff05a122c8c69c079627bb5590f36"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.093894 27315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "adb882d9297a492eba3d3c1bbb9fb205"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "6f717107fdcfd5e4c514f0e78dc2956b"
server_key_iv: "8bdff05a122c8c69c079627bb5590f36"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.097273 27315 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:58:23.099612 27341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.100718 27315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:58:23.100831 27315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "adb882d9297a492eba3d3c1bbb9fb205"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "6f717107fdcfd5e4c514f0e78dc2956b"
server_key_iv: "8bdff05a122c8c69c079627bb5590f36"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.100931 27315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:23.127017 27315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:23.127637 27315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:23.127795 27315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:23.128345 27315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:23.129230 27315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:23.129277 27315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.129347 27315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:23.129374 27315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.137492 27315 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:43915
I20260811 02:58:23.137543 27549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:43915 every 8 connection(s)
I20260811 02:58:23.138327 27315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:23.142436 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 27315
I20260811 02:58:23.142534 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:23.142778 9005 external_mini_cluster.cc:1428] Setting key 455b5b2dd7e5ffceef3edacda7e8bf41
I20260811 02:58:23.144824 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:37003
--builtin_ntp_servers=127.8.203.84:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 02:58:23.147792 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:23.139987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:46065 (local address 127.8.203.126:37003)
0811 02:58:23.140108 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:23.140112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:23.141121 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 02:58:23.142557 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:23.142560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:23.142617 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:23.142689 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:23.143755 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.144424 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.145972 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.146192 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.146272 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:23.147356 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:23.147360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:23.147363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:23.147377 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:23.147401 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:23.147411 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:23.147477 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:23.147597 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 02:58:23.147656 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:58:23.148550 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:23.140361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:37003 (local address 127.8.203.65:46065)
0811 02:58:23.140987 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:23.141025 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:23.142350 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:23.142884 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:23.142893 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:23.142989 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:23.143588 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:23.143608 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.144566 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.144569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:23.145853 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:23.145866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.146332 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.146335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:23.146535 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:23.147109 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:58:23.147110 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:23.147150 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:58:23.147212 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:23.147488 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:23.147492 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:23.147496 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:58:23.147578 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:58:23.147782 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":113,"threads_started":1}
I20260811 02:58:23.149650 27550 heartbeater.cc:347] Connected to a master server at 127.8.203.126:37003
I20260811 02:58:23.149902 27550 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:23.150484 27550 heartbeater.cc:511] Master 127.8.203.126:37003 requested a full tablet report, sending...
I20260811 02:58:23.151968 27200 ts_manager.cc:194] Registered new tserver with Master: adb882d9297a492eba3d3c1bbb9fb205 (127.8.203.65:43915)
I20260811 02:58:23.153061 27200 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.65:46065
W20260811 02:58:23.257566 27558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:23.257813 27558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:23.257850 27558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:23.261776 27558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:23.261826 27558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:23.261889 27558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:58:23.266486 27558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:23.267484 27558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:23.268827 27558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:23.278213 27570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:23.278247 27578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:23.278223 27575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:23.278383 27558 server_base.cc:1034] running on GCE node
I20260811 02:58:23.278939 27558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:23.279534 27558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:23.280682 27558 hybrid_clock.cc:648] HybridClock initialized: now 1786417103280659 us; error 42 us; skew 500 ppm
I20260811 02:58:23.282409 27558 webserver.cc:460] Webserver started at http://127.8.203.66:41157/ using document root <none> and password file <none>
I20260811 02:58:23.282948 27558 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:23.283013 27558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:23.283207 27558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:23.285178 27558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "f141045ddedf479e8bab8b0775aa2ef6"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "a15291433d7d016d8aea5f88db9c6ba2"
server_key_iv: "614c4f908f99ba244ca913fffffb37f1"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.285768 27558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "f141045ddedf479e8bab8b0775aa2ef6"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "a15291433d7d016d8aea5f88db9c6ba2"
server_key_iv: "614c4f908f99ba244ca913fffffb37f1"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.289166 27558 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:58:23.291698 27589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.292869 27558 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:23.293041 27558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "f141045ddedf479e8bab8b0775aa2ef6"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "a15291433d7d016d8aea5f88db9c6ba2"
server_key_iv: "614c4f908f99ba244ca913fffffb37f1"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.293154 27558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:23.321730 27558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:23.322399 27558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:23.322572 27558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:23.323153 27558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:23.324077 27558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:23.324124 27558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.324177 27558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:23.324208 27558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.332607 27558 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:41367
I20260811 02:58:23.332636 27808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:41367 every 8 connection(s)
I20260811 02:58:23.333521 27558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:58:23.341725 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 27558
I20260811 02:58:23.341825 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:58:23.342079 9005 external_mini_cluster.cc:1428] Setting key 8b78bb6917572b47a0c075a2f1b64188
I20260811 02:58:23.343977 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:37003
--builtin_ntp_servers=127.8.203.84:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 02:58:23.346210 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:23.335292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:49409 (local address 127.8.203.126:37003)
0811 02:58:23.335422 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:23.335426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:23.336465 (+ 1039us) server_negotiation.cc:407] Connection header received
0811 02:58:23.337922 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:23.337926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:23.337984 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:23.338059 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:23.338967 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.339502 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.340614 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.340838 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.340927 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:23.345726 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:23.345729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:23.345731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:23.345744 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:23.345766 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:23.345775 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:23.345839 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:23.345999 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 02:58:23.346068 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:58:23.347174 27813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:23.335663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:37003 (local address 127.8.203.66:49409)
0811 02:58:23.336322 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:23.336365 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:23.337708 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:23.338241 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:23.338248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:23.338332 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:23.338817 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:23.338830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.339624 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.339627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:23.340489 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:23.340499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.340979 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.340984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:23.341142 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:23.345450 (+ 4308us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:58:23.345452 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:23.345499 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:58:23.345571 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:23.345864 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:23.345872 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:23.345877 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:58:23.345991 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 02:58:23.346298 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":125,"threads_started":1}
I20260811 02:58:23.348309 27810 heartbeater.cc:347] Connected to a master server at 127.8.203.126:37003
I20260811 02:58:23.348531 27810 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:23.349071 27810 heartbeater.cc:511] Master 127.8.203.126:37003 requested a full tablet report, sending...
I20260811 02:58:23.350128 27200 ts_manager.cc:194] Registered new tserver with Master: f141045ddedf479e8bab8b0775aa2ef6 (127.8.203.66:41367)
I20260811 02:58:23.350678 27200 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.66:49409
W20260811 02:58:23.455688 27815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:23.455940 27815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:23.455991 27815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:23.459906 27815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:23.459956 27815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:23.460026 27815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:58:23.464599 27815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:23.465670 27815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:23.467012 27815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:23.476115 27826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:23.476147 27827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:23.476198 27830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:23.476449 27815 server_base.cc:1034] running on GCE node
I20260811 02:58:23.476794 27815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:23.477454 27815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:23.478605 27815 hybrid_clock.cc:648] HybridClock initialized: now 1786417103478583 us; error 34 us; skew 500 ppm
I20260811 02:58:23.480204 27815 webserver.cc:460] Webserver started at http://127.8.203.67:38921/ using document root <none> and password file <none>
I20260811 02:58:23.480746 27815 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:23.480810 27815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:23.481009 27815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:23.483017 27815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "1180d98615df4497b41c700073b19039"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "81a0ede9366f514cadf1f883ea9130ba"
server_key_iv: "f32edbacbff69a2a9c3cb760d0cd91c4"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.483552 27815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "1180d98615df4497b41c700073b19039"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "81a0ede9366f514cadf1f883ea9130ba"
server_key_iv: "f32edbacbff69a2a9c3cb760d0cd91c4"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.487061 27815 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:58:23.489452 27846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.490586 27815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:58:23.490687 27815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "1180d98615df4497b41c700073b19039"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "81a0ede9366f514cadf1f883ea9130ba"
server_key_iv: "f32edbacbff69a2a9c3cb760d0cd91c4"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.490779 27815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:23.501638 27815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:23.502198 27815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:23.502347 27815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:23.502866 27815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:23.503748 27815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:23.503808 27815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.503849 27815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:23.503870 27815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.511726 27815 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43801
I20260811 02:58:23.511750 28074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43801 every 8 connection(s)
I20260811 02:58:23.512610 27815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:58:23.520988 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 27815
I20260811 02:58:23.521083 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:58:23.521355 9005 external_mini_cluster.cc:1428] Setting key ab8ac7c31c457b6687dbd2a9c0bb1a90
I20260811 02:58:23.521436 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:23.514380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:41951 (local address 127.8.203.126:37003)
0811 02:58:23.514500 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:23.514504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:23.515485 (+ 981us) server_negotiation.cc:407] Connection header received
0811 02:58:23.516879 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:23.516882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:23.516947 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:23.517023 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:23.517943 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.518476 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.519590 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.519808 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.519897 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:23.520911 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:23.520914 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:23.520916 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:23.520929 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:23.520948 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:23.520955 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:23.521012 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:23.521196 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 02:58:23.521257 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:58:23.522337 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:23.514720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:37003 (local address 127.8.203.67:41951)
0811 02:58:23.515336 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:23.515377 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:23.516663 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:23.517206 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:23.517213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:23.517318 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:23.517782 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:23.517795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.518599 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.518602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:23.519460 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:23.519469 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:23.519948 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:23.519952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:23.520104 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:23.520643 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:58:23.520644 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:23.520683 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:58:23.520743 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:23.521023 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:23.521032 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:23.521038 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:58:23.521185 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 02:58:23.521473 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":119,"threads_started":1}
I20260811 02:58:23.523339 28077 heartbeater.cc:347] Connected to a master server at 127.8.203.126:37003
I20260811 02:58:23.523550 28077 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:23.524009 28077 heartbeater.cc:511] Master 127.8.203.126:37003 requested a full tablet report, sending...
I20260811 02:58:23.524919 27200 ts_manager.cc:194] Registered new tserver with Master: 1180d98615df4497b41c700073b19039 (127.8.203.67:43801)
I20260811 02:58:23.525506 27200 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.67:41951
I20260811 02:58:23.534809 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:58:23.537627 27275 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:23.536665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.8.203.126:37003)
0811 02:58:23.536783 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:23.536788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:23.536992 (+ 204us) server_negotiation.cc:407] Connection header received
0811 02:58:23.537115 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:23.537118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:23.537160 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:23.537223 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:23.537517 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48906: BlockingRecv error: recv got EOF from 127.0.0.1:48906 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:58:23.537906 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 27315
I20260811 02:58:23.555769 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 27558
I20260811 02:58:23.584153 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 27815
I20260811 02:58:23.608618 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 27142
2026-08-11T02:58:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (917 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T02:58:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:23Z Disabled control of system clock
I20260811 02:58:23.641160 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41089
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:41089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 02:58:23.749387 28096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:23.749617 28096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:23.749651 28096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:23.753520 28096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:23.753564 28096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:23.753577 28096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:23.753597 28096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:23.753612 28096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:23.758229 28096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:41089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:23.759124 28096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:23.760361 28096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:23.768440 28106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:23.768483 28109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:23.768491 28096 server_base.cc:1034] running on GCE node
W20260811 02:58:23.768445 28118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:23.769202 28096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:23.770226 28096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:23.771368 28096 hybrid_clock.cc:648] HybridClock initialized: now 1786417103771347 us; error 34 us; skew 500 ppm
I20260811 02:58:23.773205 28096 webserver.cc:460] Webserver started at http://127.8.203.126:41229/ using document root <none> and password file <none>
I20260811 02:58:23.773891 28096 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:23.773974 28096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:23.774209 28096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:23.776954 28096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "63a5b22df91f4013b8771772ea234f3a"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "98d391752d93d5a3c31f7783c4e26d5f"
server_key_iv: "9873f4ce6092c2aa5a64705e4901dc40"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.777701 28096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "63a5b22df91f4013b8771772ea234f3a"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "98d391752d93d5a3c31f7783c4e26d5f"
server_key_iv: "9873f4ce6092c2aa5a64705e4901dc40"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.783185 28096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:58:23.785569 28127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:23.786770 28096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:58:23.786892 28096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "63a5b22df91f4013b8771772ea234f3a"
format_stamp: "Formatted at 2026-08-11 02:58:23 on dist-test-slave-gjw7"
server_key: "98d391752d93d5a3c31f7783c4e26d5f"
server_key_iv: "9873f4ce6092c2aa5a64705e4901dc40"
server_key_version: "encryptionkey@0"
I20260811 02:58:23.786989 28096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:23.791344 28096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:23.791903 28096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:23.792044 28096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:23.798484 28096 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:41089
I20260811 02:58:23.798496 28223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:41089 every 8 connection(s)
I20260811 02:58:23.799321 28096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:23.802295 28225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:23.806694 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 28096
I20260811 02:58:23.806777 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:58:23.807006 9005 external_mini_cluster.cc:1428] Setting key b2f9bb5f07b9ff89e9355da9eec84775
I20260811 02:58:23.807703 28225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a: Bootstrap starting.
I20260811 02:58:23.809873 28225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:23.810671 28225 log.cc:824] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:23.812335 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:23.808305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52694 (local address 127.8.203.126:41089)
0811 02:58:23.808881 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:23.808896 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:23.808929 (+ 33us) server_negotiation.cc:407] Connection header received
0811 02:58:23.809587 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:23.809606 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:23.809950 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:23.810243 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:23.811088 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:23.811094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:23.811110 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:23.811124 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:23.811194 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:23.811221 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:23.811326 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:23.811430 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 02:58:23.811686 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":90,"threads_started":1}
I20260811 02:58:23.812626 28225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a: No bootstrap required, opened a new log
I20260811 02:58:23.815958 28225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a5b22df91f4013b8771772ea234f3a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41089 } }
I20260811 02:58:23.816169 28225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:23.816239 28225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a5b22df91f4013b8771772ea234f3a, State: Initialized, Role: FOLLOWER
I20260811 02:58:23.816623 28225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a5b22df91f4013b8771772ea234f3a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41089 } }
I20260811 02:58:23.816749 28225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:23.816811 28225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:23.816908 28225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:23.817759 28225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a5b22df91f4013b8771772ea234f3a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41089 } }
I20260811 02:58:23.818058 28225 leader_election.cc:302] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63a5b22df91f4013b8771772ea234f3a; no voters:
I20260811 02:58:23.818315 28225 leader_election.cc:288] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:23.818408 28235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:23.818638 28235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [term 1 LEADER]: Becoming Leader. State: Replica: 63a5b22df91f4013b8771772ea234f3a, State: Running, Role: LEADER
I20260811 02:58:23.818925 28235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a5b22df91f4013b8771772ea234f3a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41089 } }
I20260811 02:58:23.819515 28225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:23.820415 28237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63a5b22df91f4013b8771772ea234f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a5b22df91f4013b8771772ea234f3a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41089 } } }
I20260811 02:58:23.820547 28237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:23.820807 28238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63a5b22df91f4013b8771772ea234f3a. Latest consensus state: current_term: 1 leader_uuid: "63a5b22df91f4013b8771772ea234f3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a5b22df91f4013b8771772ea234f3a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41089 } } }
I20260811 02:58:23.820905 28238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:23.821002 28247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:23.823987 28247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:23.829259 28247 catalog_manager.cc:1347] Generated new cluster ID: 8cd0c11a3d6d4874beee39c3d7954bdf
I20260811 02:58:23.829342 28247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:23.843266 28247 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:58:23.843580 28247 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:23.855157 28247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63a5b22df91f4013b8771772ea234f3a: Generated new TSK 0
I20260811 02:58:23.855782 28247 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:58:23.867394 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41089
--builtin_ntp_servers=127.8.203.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 02:58:23.982532 28264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:23.982766 28264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:23.982805 28264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:23.986690 28264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:23.986740 28264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:23.986792 28264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:58:23.991330 28264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:23.992236 28264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:23.993551 28264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:58:24.002605 28264 server_base.cc:1034] running on GCE node
W20260811 02:58:24.002538 28278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:24.002545 28280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:24.002537 28274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:24.003137 28264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:24.003729 28264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:24.004873 28264 hybrid_clock.cc:648] HybridClock initialized: now 1786417104004849 us; error 37 us; skew 500 ppm
I20260811 02:58:24.006565 28264 webserver.cc:460] Webserver started at http://127.8.203.65:41355/ using document root <none> and password file <none>
I20260811 02:58:24.007089 28264 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:24.007149 28264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:24.007347 28264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:24.009284 28264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "2e545f57974343b5a1a842adc96228bc"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "84939b9943ed46718b1ac9dc4d0a9f2e"
server_key_iv: "319e4ceaac70e260acc0aaaff972e90b"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.009865 28264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "2e545f57974343b5a1a842adc96228bc"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "84939b9943ed46718b1ac9dc4d0a9f2e"
server_key_iv: "319e4ceaac70e260acc0aaaff972e90b"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.013382 28264 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:58:24.015719 28294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.016883 28264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:24.016989 28264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "2e545f57974343b5a1a842adc96228bc"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "84939b9943ed46718b1ac9dc4d0a9f2e"
server_key_iv: "319e4ceaac70e260acc0aaaff972e90b"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.017083 28264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:24.021529 28264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:24.022076 28264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:24.022233 28264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:24.022727 28264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:24.023567 28264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:24.023614 28264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.023663 28264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:24.023693 28264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.031469 28264 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:42773
I20260811 02:58:24.031499 28539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:42773 every 8 connection(s)
I20260811 02:58:24.032259 28264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:24.033989 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 28264
I20260811 02:58:24.034099 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:24.034344 9005 external_mini_cluster.cc:1428] Setting key aeb9b1b369c76c5ba130e3f66720b504
I20260811 02:58:24.036139 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41089
--builtin_ntp_servers=127.8.203.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 02:58:24.038170 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:24.033911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:41909 (local address 127.8.203.126:41089)
0811 02:58:24.034065 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:24.034070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:24.035179 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 02:58:24.036496 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:24.036500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:24.036542 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:24.036612 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:24.037688 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:24.037691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:24.037694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:24.037709 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:24.037736 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:24.037746 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:24.037806 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:24.038029 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 02:58:24.038070 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:58:24.039302 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:24.034321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41089 (local address 127.8.203.65:41909)
0811 02:58:24.035033 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:24.035083 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:24.036289 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:24.036769 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:24.036776 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:24.036861 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:24.037408 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:58:24.037409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:24.037448 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:58:24.037515 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:24.037875 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:24.037884 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:24.037889 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:58:24.038046 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 02:58:24.038373 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":122,"threads_started":1}
I20260811 02:58:24.040424 28541 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41089
I20260811 02:58:24.040630 28541 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:24.041021 28541 heartbeater.cc:511] Master 127.8.203.126:41089 requested a full tablet report, sending...
I20260811 02:58:24.042487 28149 ts_manager.cc:194] Registered new tserver with Master: 2e545f57974343b5a1a842adc96228bc (127.8.203.65:42773)
W20260811 02:58:24.145634 28544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:24.145892 28544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:24.145944 28544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:24.149849 28544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:24.149899 28544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:24.149972 28544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:58:24.154544 28544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:24.155570 28544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:24.156956 28544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:24.166388 28557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:24.166390 28561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:24.166390 28555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:24.166764 28544 server_base.cc:1034] running on GCE node
I20260811 02:58:24.167202 28544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:24.167843 28544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:24.168999 28544 hybrid_clock.cc:648] HybridClock initialized: now 1786417104168982 us; error 31 us; skew 500 ppm
I20260811 02:58:24.170739 28544 webserver.cc:460] Webserver started at http://127.8.203.66:42341/ using document root <none> and password file <none>
I20260811 02:58:24.171283 28544 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:24.171360 28544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:24.171569 28544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:24.173627 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "bbd0a5b3083349248892f6e039381d48"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "4b1d98f02d85d4eebfbe2b05d8d6b006"
server_key_iv: "2518052da20823f65c666dccc19cb132"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.174161 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "bbd0a5b3083349248892f6e039381d48"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "4b1d98f02d85d4eebfbe2b05d8d6b006"
server_key_iv: "2518052da20823f65c666dccc19cb132"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.178470 28544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:58:24.180962 28572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.182230 28544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 02:58:24.182351 28544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "bbd0a5b3083349248892f6e039381d48"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "4b1d98f02d85d4eebfbe2b05d8d6b006"
server_key_iv: "2518052da20823f65c666dccc19cb132"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.182453 28544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:24.186889 28544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:24.187469 28544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:24.187614 28544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:24.188115 28544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:24.188987 28544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:24.189033 28544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.189083 28544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:24.189109 28544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.197379 28544 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:41073
I20260811 02:58:24.197405 28794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:41073 every 8 connection(s)
I20260811 02:58:24.198199 28544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:58:24.202914 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 28544
I20260811 02:58:24.203011 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:58:24.203258 9005 external_mini_cluster.cc:1428] Setting key 6137b2da07affec49594012ff2fc9a2c
I20260811 02:58:24.204219 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:24.200057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:33997 (local address 127.8.203.126:41089)
0811 02:58:24.200186 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:24.200189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:24.201192 (+ 1003us) server_negotiation.cc:407] Connection header received
0811 02:58:24.202536 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:24.202539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:24.202589 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:24.202663 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:24.203784 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:24.203789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:24.203792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:24.203806 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:24.203830 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:24.203851 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:24.203944 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:24.204077 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 02:58:24.204115 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:58:24.205157 28798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:24.200409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41089 (local address 127.8.203.66:33997)
0811 02:58:24.201040 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:24.201080 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:24.202324 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:24.202854 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:24.202861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:24.202947 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:24.203492 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:58:24.203493 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:24.203533 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:58:24.203617 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:24.203939 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:24.203948 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:24.203954 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:58:24.204121 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:58:24.204365 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":113,"threads_started":1}
I20260811 02:58:24.205185 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41089
--builtin_ntp_servers=127.8.203.84:36589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 02:58:24.206215 28795 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41089
I20260811 02:58:24.206431 28795 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:24.206847 28795 heartbeater.cc:511] Master 127.8.203.126:41089 requested a full tablet report, sending...
I20260811 02:58:24.207782 28149 ts_manager.cc:194] Registered new tserver with Master: bbd0a5b3083349248892f6e039381d48 (127.8.203.66:41073)
W20260811 02:58:24.319329 28799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:24.319587 28799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:24.319643 28799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:24.323859 28799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:24.323910 28799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:24.323988 28799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:58:24.328601 28799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:36589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:24.329629 28799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:24.330956 28799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:24.340271 28811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:24.340274 28810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:24.340267 28814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:24.340526 28799 server_base.cc:1034] running on GCE node
I20260811 02:58:24.340893 28799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:24.341521 28799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:24.342667 28799 hybrid_clock.cc:648] HybridClock initialized: now 1786417104342641 us; error 38 us; skew 500 ppm
I20260811 02:58:24.344313 28799 webserver.cc:460] Webserver started at http://127.8.203.67:37739/ using document root <none> and password file <none>
I20260811 02:58:24.344858 28799 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:24.344920 28799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:24.345111 28799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:24.347143 28799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "6ff545db47c74f2ba32fad056b5c29c6"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "d7ce5991e61cc01f1c768f955237f4a7"
server_key_iv: "aee80a5b871a1af34166722a2b97b0c5"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.347653 28799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "6ff545db47c74f2ba32fad056b5c29c6"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "d7ce5991e61cc01f1c768f955237f4a7"
server_key_iv: "aee80a5b871a1af34166722a2b97b0c5"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.351250 28799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:58:24.353603 28826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.354807 28799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:58:24.354939 28799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "6ff545db47c74f2ba32fad056b5c29c6"
format_stamp: "Formatted at 2026-08-11 02:58:24 on dist-test-slave-gjw7"
server_key: "d7ce5991e61cc01f1c768f955237f4a7"
server_key_iv: "aee80a5b871a1af34166722a2b97b0c5"
server_key_version: "encryptionkey@0"
I20260811 02:58:24.355039 28799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:24.359464 28799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:24.360026 28799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:24.360172 28799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:24.360661 28799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:24.361541 28799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:24.361588 28799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.361637 28799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:24.361662 28799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:24.369979 28799 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:37341
I20260811 02:58:24.370021 29066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:37341 every 8 connection(s)
I20260811 02:58:24.370816 28799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:58:24.371959 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 28799
I20260811 02:58:24.372081 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:58:24.372368 9005 external_mini_cluster.cc:1428] Setting key fde473bbcc36ea35365ca5bf781dde8d
I20260811 02:58:24.376689 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:24.373979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:53963 (local address 127.8.203.126:41089)
0811 02:58:24.374116 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:24.374120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:24.374136 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:58:24.375170 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:24.375174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:24.375217 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:24.375288 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:24.376271 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:24.376275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:24.376277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:58:24.376289 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:24.376309 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:58:24.376318 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:24.376394 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:24.376521 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 02:58:24.376563 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:58:24.377539 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:24.373002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41089 (local address 127.8.203.67:53963)
0811 02:58:24.373721 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:24.373772 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:24.374981 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:24.375457 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:24.375464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:24.375549 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:24.376038 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:58:24.376039 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:24.376077 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:58:24.376142 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:24.376391 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:24.376400 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:24.376406 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:58:24.376556 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 02:58:24.376787 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":107,"threads_started":1}
I20260811 02:58:24.378497 29072 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41089
I20260811 02:58:24.378715 29072 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:24.379145 29072 heartbeater.cc:511] Master 127.8.203.126:41089 requested a full tablet report, sending...
I20260811 02:58:24.380084 28149 ts_manager.cc:194] Registered new tserver with Master: 6ff545db47c74f2ba32fad056b5c29c6 (127.8.203.67:37341)
I20260811 02:58:24.386082 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:58:24.388782 28233 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:58:24.387919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.8.203.126:41089)
0811 02:58:24.388034 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:24.388038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:24.388229 (+ 191us) server_negotiation.cc:407] Connection header received
0811 02:58:24.388318 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:24.388321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:24.388356 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:24.388421 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:24.388663 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52702: BlockingRecv error: recv got EOF from 127.0.0.1:52702 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:58:24.388943 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 28264
I20260811 02:58:24.411569 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 28544
I20260811 02:58:24.436075 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 28799
I20260811 02:58:24.447145 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 28096
2026-08-11T02:58:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (842 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:58:24 dist-test-slave-gjw7 krb5kdc[29092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:58:24 dist-test-slave-gjw7 krb5kdc[29092](info): set up 2 sockets
Aug 11 02:58:24 dist-test-slave-gjw7 krb5kdc[29092](info): commencing operation
krb5kdc: starting...
W20260811 02:58:25.652419 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:58:25 dist-test-slave-gjw7 krb5kdc[29092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:58:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:58:25.764560 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41405
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:25.872099 29117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:25.872337 29117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:25.872400 29117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:25.876318 29117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:58:25.876377 29117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:25.876405 29117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:25.876435 29117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:58:25.876461 29117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:58:25.882915 29117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:41405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:25.884294 29117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:25.885617 29117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:25.896616 29123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:25.896672 29137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:25.896610 29127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:25.896914 29117 server_base.cc:1034] running on GCE node
I20260811 02:58:25.897508 29117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:25.898685 29117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:25.899893 29117 hybrid_clock.cc:648] HybridClock initialized: now 1786417105899824 us; error 79 us; skew 500 ppm
Aug 11 02:58:25 dist-test-slave-gjw7 krb5kdc[29092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417105, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:25.904522 29117 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:58:25.906648 29117 webserver.cc:460] Webserver started at http://127.8.203.126:38941/ using document root <none> and password file <none>
I20260811 02:58:25.907318 29117 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:25.907405 29117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:25.907658 29117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:25.910427 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a"
format_stamp: "Formatted at 2026-08-11 02:58:25 on dist-test-slave-gjw7"
server_key: "d07be230534058852c70b723590c8bc2"
server_key_iv: "e180d9e1efe2593535673683f5b1e958"
server_key_version: "encryptionkey@0"
I20260811 02:58:25.911116 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a"
format_stamp: "Formatted at 2026-08-11 02:58:25 on dist-test-slave-gjw7"
server_key: "d07be230534058852c70b723590c8bc2"
server_key_iv: "e180d9e1efe2593535673683f5b1e958"
server_key_version: "encryptionkey@0"
I20260811 02:58:25.918346 29117 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 02:58:25.921087 29154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:25.922299 29117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:58:25.922433 29117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a"
format_stamp: "Formatted at 2026-08-11 02:58:25 on dist-test-slave-gjw7"
server_key: "d07be230534058852c70b723590c8bc2"
server_key_iv: "e180d9e1efe2593535673683f5b1e958"
server_key_version: "encryptionkey@0"
I20260811 02:58:25.922529 29117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:25.937911 29117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:25.940436 29117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:25.940591 29117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:25.947086 29117 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:41405
I20260811 02:58:25.947340 29268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:41405 every 8 connection(s)
I20260811 02:58:25.948004 29117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:58:25.949699 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 29117
I20260811 02:58:25.949822 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:58:25.950131 9005 external_mini_cluster.cc:1428] Setting key fa51c81a796a72af065a9d097326a1e8
I20260811 02:58:25.951009 29271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:25.956499 29271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a: Bootstrap starting.
Aug 11 02:58:25 dist-test-slave-gjw7 krb5kdc[29092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:25.958896 29271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:25.959695 29271 log.cc:824] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:25.961501 29271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a: No bootstrap required, opened a new log
I20260811 02:58:25.963076 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:25.951528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.8.203.126:41405)
0811 02:58:25.951955 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:25.951970 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:25.952005 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:58:25.952632 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:25.952648 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:25.952966 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:25.953261 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:25.954068 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:25.955057 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:25.955891 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:25.956361 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:25.956506 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:25.958961 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:25.958968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:25.958986 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:25.959018 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:25.960495 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:25.960892 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:25.960897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:25.960901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:25.961008 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:25.961254 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:25.961257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:25.961259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:25.961694 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:25.961868 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:25.962137 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 02:58:25.962407 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":136,"threads_started":1}
I20260811 02:58:25.964839 29271 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 02:58:25.965085 29271 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:25.965148 29271 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d7887fc2ab4f7ea8b75d38e29d169a, State: Initialized, Role: FOLLOWER
I20260811 02:58:25.965545 29271 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 02:58:25.965677 29271 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:58:25.965723 29271 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:58:25.965791 29271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:25.966547 29271 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 02:58:25.966833 29271 leader_election.cc:302] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1d7887fc2ab4f7ea8b75d38e29d169a; no voters:
I20260811 02:58:25.967072 29271 leader_election.cc:288] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:58:25.967163 29286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:25.967365 29286 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [term 1 LEADER]: Becoming Leader. State: Replica: b1d7887fc2ab4f7ea8b75d38e29d169a, State: Running, Role: LEADER
I20260811 02:58:25.967657 29286 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 02:58:25.968149 29271 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:58:25.969244 29290 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1d7887fc2ab4f7ea8b75d38e29d169a. Latest consensus state: current_term: 1 leader_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } } }
I20260811 02:58:25.969395 29290 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [sys.catalog]: This master's current role is: LEADER
I20260811 02:58:25.969616 29289 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1d7887fc2ab4f7ea8b75d38e29d169a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } } }
I20260811 02:58:25.969699 29289 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a [sys.catalog]: This master's current role is: LEADER
W20260811 02:58:25.972405 29315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:58:25.972467 29315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:58:25.972537 29302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:58:25.973168 29302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:58:25.978138 29302 catalog_manager.cc:1347] Generated new cluster ID: f1a05e3557e346f687e8aae0874e78ff
I20260811 02:58:25.978196 29302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:58:25.996670 29302 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:58:25.997531 29302 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:58:26.003749 29302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a: Generated new TSK 0
I20260811 02:58:26.004305 29302 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:58:26.031539 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41405
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:26.139017 29326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:26.139245 29326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:26.139292 29326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:26.143555 29326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:26.143605 29326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:26.143671 29326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:58:26.148581 29326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:26.149530 29326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:26.150496 29326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:26.159438 29341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:26.159433 29337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:26.159435 29338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:26.159484 29326 server_base.cc:1034] running on GCE node
I20260811 02:58:26.160032 29326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:26.160586 29326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:26.161731 29326 hybrid_clock.cc:648] HybridClock initialized: now 1786417106161713 us; error 29 us; skew 500 ppm
Aug 11 02:58:26 dist-test-slave-gjw7 krb5kdc[29092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417106, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:26.164239 29326 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:58:26.165051 29326 webserver.cc:460] Webserver started at http://127.8.203.65:41317/ using document root <none> and password file <none>
I20260811 02:58:26.165608 29326 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:26.165669 29326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:26.165858 29326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:26.167802 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "0a2a7141355b4da999035a79452b7449"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "2b3066a448885afe07a082882be96d86"
server_key_iv: "dac61d28cddd47e2b9991a3d8e707df6"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.168345 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "0a2a7141355b4da999035a79452b7449"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "2b3066a448885afe07a082882be96d86"
server_key_iv: "dac61d28cddd47e2b9991a3d8e707df6"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.172151 29326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 02:58:26.174427 29357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.175554 29326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:58:26.175647 29326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "0a2a7141355b4da999035a79452b7449"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "2b3066a448885afe07a082882be96d86"
server_key_iv: "dac61d28cddd47e2b9991a3d8e707df6"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.175738 29326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:26.198803 29326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:26.201392 29326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:26.201553 29326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:26.202102 29326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:26.202880 29326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:26.202924 29326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.202965 29326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:26.202994 29326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.211259 29326 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:44947
I20260811 02:58:26.211282 29591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:44947 every 8 connection(s)
I20260811 02:58:26.212086 29326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:58:26.217533 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 29326
I20260811 02:58:26.217633 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:58:26.217907 9005 external_mini_cluster.cc:1428] Setting key 011a4c8e62a270d42d8aa8a201c347ac
Aug 11 02:58:26 dist-test-slave-gjw7 krb5kdc[29092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417106, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:26.225407 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.213718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:44297 (local address 127.8.203.126:41405)
0811 02:58:26.213873 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.213877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.214693 (+ 816us) server_negotiation.cc:407] Connection header received
0811 02:58:26.215987 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.215990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.216038 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.216109 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:26.217190 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.217799 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.219161 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.219409 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.219501 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.221974 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:26.221976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:26.221978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:26.222002 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:26.223331 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.223809 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.223812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.223813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.223893 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.224292 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.224294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.224295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.224479 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:26.224552 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.225126 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.225229 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:58:26.226485 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.214104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41405 (local address 127.8.203.65:44297)
0811 02:58:26.214549 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.214591 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.215773 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.216268 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.216275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.216623 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:26.217073 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.217087 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.217926 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.217930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.219047 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.219058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.219557 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.219561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.219720 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.220221 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:26.220237 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:26.221806 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:26.223449 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:26.223456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:26.223470 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:26.223715 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:26.224012 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:26.224016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:26.224018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:26.224202 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:26.224577 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:26.224583 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:26.224891 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.225108 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.225465 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":115,"threads_started":1}
I20260811 02:58:26.227716 29592 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41405
I20260811 02:58:26.227933 29592 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:26.228389 29592 heartbeater.cc:511] Master 127.8.203.126:41405 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:58:26.229853 29182 ts_manager.cc:194] Registered new tserver with Master: 0a2a7141355b4da999035a79452b7449 (127.8.203.65:44947)
I20260811 02:58:26.231000 29182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:44297
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:58:26.245275 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41405
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:26.353905 29601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:26.354130 29601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:26.354173 29601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:26.358063 29601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:26.358109 29601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:26.358173 29601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:58:26.363125 29601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:26.364102 29601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:26.365034 29601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:26.374131 29614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:26.374157 29619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:26.374159 29610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:26.374312 29601 server_base.cc:1034] running on GCE node
I20260811 02:58:26.374848 29601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:26.375420 29601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:26.376565 29601 hybrid_clock.cc:648] HybridClock initialized: now 1786417106376542 us; error 39 us; skew 500 ppm
Aug 11 02:58:26 dist-test-slave-gjw7 krb5kdc[29092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417106, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:26.379108 29601 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:58:26.380039 29601 webserver.cc:460] Webserver started at http://127.8.203.66:33129/ using document root <none> and password file <none>
I20260811 02:58:26.380604 29601 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:26.380669 29601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:26.380867 29601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:26.382887 29601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "b8fdcfe6d2704fc98538d00e8ea65382"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "e526c5f5d0190b8d8532bb89dfbcfdaa"
server_key_iv: "173faf28f7cc02f9933587a66226f0e1"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.383412 29601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "b8fdcfe6d2704fc98538d00e8ea65382"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "e526c5f5d0190b8d8532bb89dfbcfdaa"
server_key_iv: "173faf28f7cc02f9933587a66226f0e1"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.386945 29601 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:58:26.389120 29629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.390262 29601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:26.390367 29601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "b8fdcfe6d2704fc98538d00e8ea65382"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "e526c5f5d0190b8d8532bb89dfbcfdaa"
server_key_iv: "173faf28f7cc02f9933587a66226f0e1"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.390468 29601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:26.431284 29601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:26.434265 29601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:26.434439 29601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:26.435024 29601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:26.435883 29601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:26.435952 29601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.436003 29601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:26.436028 29601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.445204 29601 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:46121
I20260811 02:58:26.445219 29852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:46121 every 8 connection(s)
I20260811 02:58:26.446146 29601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:58:26.451447 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 29601
I20260811 02:58:26.451561 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:58:26.451846 9005 external_mini_cluster.cc:1428] Setting key cf0cefdffa3321a7af1891a3f596d780
Aug 11 02:58:26 dist-test-slave-gjw7 krb5kdc[29092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417106, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:26.459466 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.447927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:39883 (local address 127.8.203.126:41405)
0811 02:58:26.448067 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.448071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.448895 (+ 824us) server_negotiation.cc:407] Connection header received
0811 02:58:26.450234 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.450237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.450294 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.450382 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:26.451495 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.452086 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.453341 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.453573 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.453662 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.455960 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:26.455963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:26.455965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:26.455987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:26.457414 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.457914 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.457918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.457919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.458013 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.458355 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.458359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.458360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.458596 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:26.458699 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.459162 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.459290 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:58:26.460178 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.448294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41405 (local address 127.8.203.66:39883)
0811 02:58:26.448753 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.448794 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.450028 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.450551 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.450558 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.450901 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:26.451356 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.451370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.452200 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.452203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.453208 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.453219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.453707 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.453711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.453868 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.454431 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:26.454445 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:26.455794 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:26.457526 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:26.457531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:26.457542 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:26.457810 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:26.458099 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:26.458101 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:26.458103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:26.458256 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:26.458699 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:26.458704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:26.458927 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.459085 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.459363 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":133,"threads_started":1}
I20260811 02:58:26.461282 29853 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41405
I20260811 02:58:26.461534 29853 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:26.462004 29853 heartbeater.cc:511] Master 127.8.203.126:41405 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 02:58:26.463013 29182 ts_manager.cc:194] Registered new tserver with Master: b8fdcfe6d2704fc98538d00e8ea65382 (127.8.203.66:46121)
I20260811 02:58:26.463655 29182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:39883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:58:26.478205 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41405
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:58:26.587980 29863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:58:26.588212 29863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:58:26.588256 29863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:58:26.592152 29863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:58:26.592199 29863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:58:26.592268 29863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:58:26.597455 29863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:58:26.598378 29863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:58:26.599313 29863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:58:26.608609 29873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:58:26.608605 29872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:26.608690 29863 server_base.cc:1034] running on GCE node
W20260811 02:58:26.608608 29878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:58:26.609159 29863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:58:26.609761 29863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:58:26.610926 29863 hybrid_clock.cc:648] HybridClock initialized: now 1786417106610909 us; error 26 us; skew 500 ppm
Aug 11 02:58:26 dist-test-slave-gjw7 krb5kdc[29092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417106, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:58:26.613461 29863 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:58:26.614243 29863 webserver.cc:460] Webserver started at http://127.8.203.67:40707/ using document root <none> and password file <none>
I20260811 02:58:26.614771 29863 fs_manager.cc:359] Metadata directory not provided
I20260811 02:58:26.614835 29863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:58:26.615042 29863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:58:26.616990 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "38a4025b106e497c8f11dbe99589f53d"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "193025b2de4a371e904f1e4ab3c26e10"
server_key_iv: "4f12fa714cf961fb6c9e4c8e27ce6962"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.617542 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "38a4025b106e497c8f11dbe99589f53d"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "193025b2de4a371e904f1e4ab3c26e10"
server_key_iv: "4f12fa714cf961fb6c9e4c8e27ce6962"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.620961 29863 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:58:26.623211 29891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.624346 29863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:58:26.624456 29863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "38a4025b106e497c8f11dbe99589f53d"
format_stamp: "Formatted at 2026-08-11 02:58:26 on dist-test-slave-gjw7"
server_key: "193025b2de4a371e904f1e4ab3c26e10"
server_key_iv: "4f12fa714cf961fb6c9e4c8e27ce6962"
server_key_version: "encryptionkey@0"
I20260811 02:58:26.624547 29863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:58:26.655650 29863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:58:26.658519 29863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:58:26.658697 29863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:58:26.659291 29863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:58:26.660131 29863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:58:26.660178 29863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.660233 29863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:58:26.660264 29863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:58:26.668699 29863 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:35717
I20260811 02:58:26.668735 30156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:35717 every 8 connection(s)
I20260811 02:58:26.669661 29863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:58:26.674315 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 29863
I20260811 02:58:26.674427 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:58:26.674712 9005 external_mini_cluster.cc:1428] Setting key 331a0f98f4601d34ba65346099e8443a
Aug 11 02:58:26 dist-test-slave-gjw7 krb5kdc[29092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417106, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:58:26.682837 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.671526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:35963 (local address 127.8.203.126:41405)
0811 02:58:26.671697 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.671702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.672574 (+ 872us) server_negotiation.cc:407] Connection header received
0811 02:58:26.673898 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.673901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.673960 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.674038 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:26.675385 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.675910 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.677027 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.677248 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.677379 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.679709 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:26.679712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:26.679714 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:26.679736 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:26.680931 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.681388 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.681392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.681394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.681474 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.681780 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.681783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.681785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.681969 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:26.682065 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.682554 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.682676 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:58:26.683588 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.671948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41405 (local address 127.8.203.67:35963)
0811 02:58:26.672423 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.672466 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.673676 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.674222 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.674230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.674640 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:58:26.675244 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.675262 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.676036 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.676039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.676907 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.676917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.677431 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.677435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.677601 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.678080 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:58:26.678096 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:58:26.679562 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:58:26.681033 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:26.681039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:26.681054 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:26.681285 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:26.681553 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:58:26.681556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:58:26.681558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:58:26.681695 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:58:26.682074 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:58:26.682081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:58:26.682313 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.682495 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.682801 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":125,"threads_started":1}
I20260811 02:58:26.684545 30158 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41405
I20260811 02:58:26.684757 30158 heartbeater.cc:464] Registering TS with master...
I20260811 02:58:26.685202 30158 heartbeater.cc:511] Master 127.8.203.126:41405 requested a full tablet report, sending...
I20260811 02:58:26.686161 29182 ts_manager.cc:194] Registered new tserver with Master: 38a4025b106e497c8f11dbe99589f53d (127.8.203.67:35717)
I20260811 02:58:26.686672 29182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:35963
I20260811 02:58:26.688270 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:58:26.696440 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.690066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.8.203.126:41405)
0811 02:58:26.690165 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.690170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.690375 (+ 205us) server_negotiation.cc:407] Connection header received
0811 02:58:26.690494 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.690497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.690535 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.690594 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:58:26.691244 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.691723 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.692526 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.692725 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.692809 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.694147 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:58:26.694150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:58:26.694152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:58:26.694173 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:58:26.695228 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.695573 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.695575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.695576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.695636 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:58:26.695807 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:58:26.695809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:58:26.695810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:58:26.696004 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:58:26.696095 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.696251 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.696333 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 02:58:26.698808 29182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:58:26.701695 29182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:58:26.715134 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.709200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:44947 (local address 127.0.0.1:44502)
0811 02:58:26.710346 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.710357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.710425 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.711059 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.711062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.711090 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:26.711213 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.711221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.712367 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.712370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.713586 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.713596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.714607 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.714611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.714724 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.714768 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.714873 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.714989 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":46,"spinlock_wait_cycles":768,"thread_start_us":454,"threads_started":1}
I20260811 02:58:26.715150 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.709421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.8.203.65:44947)
0811 02:58:26.710546 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.710575 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.710601 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:58:26.710695 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.710701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.710909 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.711352 (+ 443us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:26.711509 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.712241 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.713700 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.714481 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.714593 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.714742 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.714914 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.715006 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":86,"threads_started":1}
I20260811 02:58:26.715330 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.709582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:46121 (local address 127.0.0.1:34068)
0811 02:58:26.710472 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.710485 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.710571 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.711360 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.711363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.711379 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:26.711468 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.711474 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.712816 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.712819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.714004 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.714012 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.714978 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.714981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.715071 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.715092 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.715153 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.715215 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":91,"threads_started":1}
I20260811 02:58:26.715656 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.710040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34068 (local address 127.8.203.66:46121)
0811 02:58:26.710798 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.710812 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.710832 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:58:26.710901 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.710905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.711064 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.711192 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:26.711591 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.712675 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.714110 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.714676 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.714774 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.714917 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.715420 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.715515 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":107,"threads_started":1}
I20260811 02:58:26.715812 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.710035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.8.203.67:35717)
0811 02:58:26.710626 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.710636 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.710653 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:58:26.710717 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.710721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.710862 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.711008 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:26.711271 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.712278 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.713892 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.714654 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.714743 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.714842 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.715438 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.715687 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":121,"threads_started":1}
I20260811 02:58:26.716027 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.709346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:35717 (local address 127.0.0.1:36304)
0811 02:58:26.710142 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.710189 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.710310 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.711016 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.711022 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.711044 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:26.711171 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.711177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.712421 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.712424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.713765 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.713777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.714909 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.714912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.714998 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.715019 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.715088 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.715152 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":41,"spinlock_wait_cycles":1920,"thread_start_us":95,"threads_started":1}
I20260811 02:58:26.718247 29454 tablet_service.cc:1467] Processing CreateTablet for tablet dff092b0af4041ae9b1dd36ad0641afc (DEFAULT_TABLE table=test-table [id=3c02cdc4cecc43d8bb7368273bf10a5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.718519 29717 tablet_service.cc:1467] Processing CreateTablet for tablet dff092b0af4041ae9b1dd36ad0641afc (DEFAULT_TABLE table=test-table [id=3c02cdc4cecc43d8bb7368273bf10a5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.718622 30010 tablet_service.cc:1467] Processing CreateTablet for tablet dff092b0af4041ae9b1dd36ad0641afc (DEFAULT_TABLE table=test-table [id=3c02cdc4cecc43d8bb7368273bf10a5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:58:26.719214 29454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff092b0af4041ae9b1dd36ad0641afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.719566 29717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff092b0af4041ae9b1dd36ad0641afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.719573 30010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff092b0af4041ae9b1dd36ad0641afc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:58:26.725579 30191 tablet_bootstrap.cc:492] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449: Bootstrap starting.
I20260811 02:58:26.726480 30195 tablet_bootstrap.cc:492] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Bootstrap starting.
I20260811 02:58:26.727200 30197 tablet_bootstrap.cc:492] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Bootstrap starting.
I20260811 02:58:26.728242 30191 tablet_bootstrap.cc:654] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.729120 30191 log.cc:824] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:26.729745 30197 tablet_bootstrap.cc:654] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.729846 30195 tablet_bootstrap.cc:654] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Neither blocks nor log segments found. Creating new log.
I20260811 02:58:26.730502 30197 log.cc:824] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:26.730607 30195 log.cc:824] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:58:26.731499 30191 tablet_bootstrap.cc:492] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449: No bootstrap required, opened a new log
I20260811 02:58:26.731690 30191 ts_tablet_manager.cc:1397] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:58:26.732218 30195 tablet_bootstrap.cc:492] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: No bootstrap required, opened a new log
I20260811 02:58:26.732376 30197 tablet_bootstrap.cc:492] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: No bootstrap required, opened a new log
I20260811 02:58:26.732385 30195 ts_tablet_manager.cc:1397] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 02:58:26.732556 30197 ts_tablet_manager.cc:1397] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:58:26.735143 30195 raft_consensus.cc:348] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.735399 30195 raft_consensus.cc:374] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.735457 30195 raft_consensus.cc:729] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a4025b106e497c8f11dbe99589f53d, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.735749 30191 raft_consensus.cc:348] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.735885 30195 consensus_queue.cc:260] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.736021 30191 raft_consensus.cc:374] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.736089 30191 raft_consensus.cc:729] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a2a7141355b4da999035a79452b7449, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.736085 30197 raft_consensus.cc:348] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.736310 30197 raft_consensus.cc:374] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:58:26.736368 30197 raft_consensus.cc:729] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8fdcfe6d2704fc98538d00e8ea65382, State: Initialized, Role: FOLLOWER
I20260811 02:58:26.736516 30191 consensus_queue.cc:260] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.736776 30195 ts_tablet_manager.cc:1428] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20260811 02:58:26.736745 30197 consensus_queue.cc:260] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.737125 30158 heartbeater.cc:503] Master 127.8.203.126:41405 was elected leader, sending a full tablet report...
I20260811 02:58:26.737147 29592 heartbeater.cc:503] Master 127.8.203.126:41405 was elected leader, sending a full tablet report...
I20260811 02:58:26.737470 30191 ts_tablet_manager.cc:1428] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:58:26.737587 30197 ts_tablet_manager.cc:1428] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:58:26.737867 29853 heartbeater.cc:503] Master 127.8.203.126:41405 was elected leader, sending a full tablet report...
I20260811 02:58:26.826160 30216 raft_consensus.cc:482] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:58:26.826385 30216 raft_consensus.cc:504] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.827533 30216 leader_election.cc:288] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a2a7141355b4da999035a79452b7449 (127.8.203.65:44947), 38a4025b106e497c8f11dbe99589f53d (127.8.203.67:35717)
I20260811 02:58:26.831523 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.827768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:44947 (local address 127.8.203.66:49329)
0811 02:58:26.827943 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.827957 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.828048 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.828344 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.828347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.828380 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:26.828504 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.828512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.829415 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.829419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.830343 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.830356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.831195 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.831198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.831278 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.831300 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.831360 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.831413 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":1,"spinlock_wait_cycles":896}
I20260811 02:58:26.831593 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.827888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:49329 (local address 127.8.203.65:44947)
0811 02:58:26.828015 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.828020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.828036 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:58:26.828188 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.828191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.828245 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.828325 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:26.828617 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.829315 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.830469 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.831086 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.831196 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.831237 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.831396 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.831469 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:58:26.831724 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.827778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:35717 (local address 127.8.203.66:44425)
0811 02:58:26.828266 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.828278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:58:26.828364 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:58:26.828610 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:58:26.828612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:58:26.828628 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:58:26.828736 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.828742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.829518 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.829520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.830397 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:58:26.830407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.831326 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.831329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:58:26.831443 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.831465 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:58:26.831536 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 02:58:26.831597 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":146,"thread_start_us":154,"threads_started":1}
I20260811 02:58:26.831773 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.827933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:44425 (local address 127.8.203.67:35717)
0811 02:58:26.828060 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.828064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.828358 (+ 294us) server_negotiation.cc:407] Connection header received
0811 02:58:26.828470 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.828474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.828523 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.828595 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:58:26.828838 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.829424 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.830531 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.831102 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.831184 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.831238 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.831578 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.831662 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:58:26.832290 29491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff092b0af4041ae9b1dd36ad0641afc" candidate_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2a7141355b4da999035a79452b7449" is_pre_election: true
I20260811 02:58:26.832341 30052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff092b0af4041ae9b1dd36ad0641afc" candidate_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a4025b106e497c8f11dbe99589f53d" is_pre_election: true
I20260811 02:58:26.832597 29491 raft_consensus.cc:2454] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8fdcfe6d2704fc98538d00e8ea65382 in term 0.
I20260811 02:58:26.832597 30052 raft_consensus.cc:2454] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8fdcfe6d2704fc98538d00e8ea65382 in term 0.
I20260811 02:58:26.833060 29631 leader_election.cc:302] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a2a7141355b4da999035a79452b7449, 38a4025b106e497c8f11dbe99589f53d, b8fdcfe6d2704fc98538d00e8ea65382; no voters:
I20260811 02:58:26.833357 30216 raft_consensus.cc:2781] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:58:26.833437 30216 raft_consensus.cc:482] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:58:26.833467 30216 raft_consensus.cc:3037] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.834472 30216 raft_consensus.cc:504] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.834779 30216 leader_election.cc:288] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [CANDIDATE]: Term 1 election: Requested vote from peers 0a2a7141355b4da999035a79452b7449 (127.8.203.65:44947), 38a4025b106e497c8f11dbe99589f53d (127.8.203.67:35717)
I20260811 02:58:26.835101 30052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff092b0af4041ae9b1dd36ad0641afc" candidate_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a4025b106e497c8f11dbe99589f53d"
I20260811 02:58:26.835101 29491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff092b0af4041ae9b1dd36ad0641afc" candidate_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a2a7141355b4da999035a79452b7449"
I20260811 02:58:26.835218 30052 raft_consensus.cc:3037] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.835218 29491 raft_consensus.cc:3037] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:58:26.836462 30052 raft_consensus.cc:2454] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8fdcfe6d2704fc98538d00e8ea65382 in term 1.
I20260811 02:58:26.836460 29491 raft_consensus.cc:2454] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8fdcfe6d2704fc98538d00e8ea65382 in term 1.
I20260811 02:58:26.836735 29634 leader_election.cc:302] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a2a7141355b4da999035a79452b7449, 38a4025b106e497c8f11dbe99589f53d, b8fdcfe6d2704fc98538d00e8ea65382; no voters:
I20260811 02:58:26.836905 30216 raft_consensus.cc:2781] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:58:26.837109 30216 raft_consensus.cc:686] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 1 LEADER]: Becoming Leader. State: Replica: b8fdcfe6d2704fc98538d00e8ea65382, State: Running, Role: LEADER
I20260811 02:58:26.837448 30216 consensus_queue.cc:237] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } }
I20260811 02:58:26.840493 29175 catalog_manager.cc:5495] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8fdcfe6d2704fc98538d00e8ea65382 (127.8.203.66). New cstate: current_term: 1 leader_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8fdcfe6d2704fc98538d00e8ea65382" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46121 } health_report { overall_health: HEALTHY } } }
I20260811 02:58:26.867357 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.864117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.8.203.66:46121)
0811 02:58:26.864229 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 02:58:26.864233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:58:26.864246 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:58:26.864386 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:58:26.864388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:58:26.864436 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:58:26.864524 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:58:26.864760 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.865224 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.866067 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:58:26.866273 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:58:26.866357 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:58:26.866639 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:58:26.867023 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:58:26.867124 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:58:26.867195 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 02:58:26.867245 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:58:26.872949 30052 raft_consensus.cc:1261] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 1 FOLLOWER]: Refusing update from remote peer b8fdcfe6d2704fc98538d00e8ea65382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:26.872949 29491 raft_consensus.cc:1261] T dff092b0af4041ae9b1dd36ad0641afc P 0a2a7141355b4da999035a79452b7449 [term 1 FOLLOWER]: Refusing update from remote peer b8fdcfe6d2704fc98538d00e8ea65382: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:58:26.873741 30224 consensus_queue.cc:1035] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a2a7141355b4da999035a79452b7449" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:26.873914 30216 consensus_queue.cc:1035] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a4025b106e497c8f11dbe99589f53d" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:58:26.881116 30231 mvcc.cc:205] Tried to move back new op lower bound from 7317164469744279552 to 7317164469608488960. Current Snapshot: MvccSnapshot[applied={T|T < 7317164469744279552}]
I20260811 02:58:26.881168 30228 mvcc.cc:205] Tried to move back new op lower bound from 7317164469744279552 to 7317164469608488960. Current Snapshot: MvccSnapshot[applied={T|T < 7317164469744279552}]
I20260811 02:58:26.889125 29183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:26.889315 29183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:58:26.891641 29183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1d7887fc2ab4f7ea8b75d38e29d169a: Sending DeleteTablet for 3 replicas of tablet dff092b0af4041ae9b1dd36ad0641afc
I20260811 02:58:26.892383 29454 tablet_service.cc:1514] Processing DeleteTablet for tablet dff092b0af4041ae9b1dd36ad0641afc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:26 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:44502
I20260811 02:58:26.892387 30010 tablet_service.cc:1514] Processing DeleteTablet for tablet dff092b0af4041ae9b1dd36ad0641afc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:26 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:36304
I20260811 02:58:26.892813 30250 tablet_replica.cc:330] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: stopping tablet replica
I20260811 02:58:26.892963 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 29326
I20260811 02:58:26.893038 29717 tablet_service.cc:1514] Processing DeleteTablet for tablet dff092b0af4041ae9b1dd36ad0641afc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:58:26 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:34068
I20260811 02:58:26.898312 30250 raft_consensus.cc:2229] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:58:26.898569 30250 raft_consensus.cc:2258] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:26.903846 30250 ts_tablet_manager.cc:1905] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:26.904175 30253 tablet_replica.cc:330] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: stopping tablet replica
I20260811 02:58:26.904436 30253 raft_consensus.cc:2229] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:58:26.904749 30253 raft_consensus.cc:2258] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:58:26.906450 30253 ts_tablet_manager.cc:1905] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:58:26.907066 30250 ts_tablet_manager.cc:1918] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:26.907191 30250 log.cc:1196] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/dff092b0af4041ae9b1dd36ad0641afc
I20260811 02:58:26.907547 30250 ts_tablet_manager.cc:1939] T dff092b0af4041ae9b1dd36ad0641afc P 38a4025b106e497c8f11dbe99589f53d: Deleting consensus metadata
I20260811 02:58:26.908543 30253 ts_tablet_manager.cc:1918] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:58:26.908635 30253 log.cc:1196] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/dff092b0af4041ae9b1dd36ad0641afc
I20260811 02:58:26.908758 29155 catalog_manager.cc:4841] TS 38a4025b106e497c8f11dbe99589f53d (127.8.203.67:35717): tablet dff092b0af4041ae9b1dd36ad0641afc (table test-table [id=3c02cdc4cecc43d8bb7368273bf10a5f]) successfully deleted
I20260811 02:58:26.908928 30253 ts_tablet_manager.cc:1939] T dff092b0af4041ae9b1dd36ad0641afc P b8fdcfe6d2704fc98538d00e8ea65382: Deleting consensus metadata
I20260811 02:58:26.909864 29156 catalog_manager.cc:4841] TS b8fdcfe6d2704fc98538d00e8ea65382 (127.8.203.66:46121): tablet dff092b0af4041ae9b1dd36ad0641afc (table test-table [id=3c02cdc4cecc43d8bb7368273bf10a5f]) successfully deleted
W20260811 02:58:26.914937 29157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:44947 (error 108)
I20260811 02:58:26.915750 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 29601
I20260811 02:58:26.915868 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:58:26.915607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:44947 (local address 127.0.0.1:44508)
0811 02:58:26.915719 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 02:58:26.915778 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:44947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 02:58:26.926093 29157 catalog_manager.cc:4584] TS 0a2a7141355b4da999035a79452b7449 (127.8.203.65:44947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dff092b0af4041ae9b1dd36ad0641afc: Network error: Client connection negotiation failed: client connection to 127.8.203.65:44947: connect: Connection refused (error 111)
I20260811 02:58:26.941095 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 29863
I20260811 02:58:26.954298 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 29117
2026-08-11T02:58:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2505 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:58:27 dist-test-slave-gjw7 krb5kdc[30263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:58:27 dist-test-slave-gjw7 krb5kdc[30263](info): set up 2 sockets
Aug 11 02:58:27 dist-test-slave-gjw7 krb5kdc[30263](info): commencing operation
krb5kdc: starting...
W20260811 02:58:28.166664 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:58:28 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:58:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:58:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/rangeradmin_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/rangeradmin_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/rangerlookup_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/rangerlookup_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
I20260811 02:58:28.323930 9005 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1jpVEs/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/postgres -l logfile start
2026-08-11 02:58:30.475 UTC [30341] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 02:58:30.476 UTC [30341] LOG: listening on IPv4 address "127.8.203.84", port 37825
2026-08-11 02:58:30.478 UTC [30341] LOG: listening on Unix socket "/tmp/.s.PGSQL.37825"
2026-08-11 02:58:30.506 UTC [30346] LOG: database system was shut down at 2026-08-11 02:58:29 UTC
2026-08-11 02:58:30.509 UTC [30341] LOG: database system is ready to accept connections
I20260811 02:58:31.618798 9005 mini_postgres.cc:96] Postgres bound to 37825
2026-08-11 02:58:31.627 UTC [30360] FATAL: database "slave" does not exist
127.8.203.84:37825 - accepting connections
I20260811 02:58:31.628453 9005 mini_ranger.cc:161] Starting Ranger...
I20260811 02:58:31.644191 9005 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 02:58:32.022339 9005 mini_ranger.cc:87] Created ranger Postgres database
I20260811 02:58:32.022437 9005 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-admin
2026-08-11 02:58:32,337 [I] DB FLAVOR :POSTGRES
2026-08-11 02:58:32,337 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:58:32,337 [I] Checking connection..
2026-08-11 02:58:32,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 02:58:32,581 [I] Checking connection passed.
2026-08-11 02:58:32,581 [I] --------- Verifying version history table ---------
2026-08-11 02:58:32,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:58:32,833 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:58:32,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:58:33,075 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:58:33,076 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 02:58:33,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 02:58:33.337 UTC [30494] WARNING: there is no transaction in progress
2026-08-11 02:58:33,346 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 02:58:33,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:58:33,568 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 02:58:33,569 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:58:33,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 02:58:33,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 02:58:34,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-11 02:58:34,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:58:34,560 [I] Table x_portal_user does not exist in database ranger
2026-08-11 02:58:34,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:58:34,805 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 02:58:34,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:58:35,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:58:35,272 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 02:58:35,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 02:58:36.125 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.139 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.153 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.166 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.179 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.194 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.244 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.250 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.262 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.271 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.280 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.291 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.300 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.305 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.311 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36.316 UTC [30707] WARNING: there is no transaction in progress
2026-08-11 02:58:36,832 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 02:58:36,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 02:58:37,078 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 02:58:37,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:58:37,329 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 02:58:37,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:58:37,566 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 02:58:37,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:58:37,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:58:38,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-11 02:58:38,292 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 02:58:38,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 02:58:38,544 [I] DB_PATCHES have already been applied
I20260811 02:58:38.550315 9005 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-admin:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1jpVEs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 02:58:38.550383 9005 mini_ranger.cc:193] Using host: 127.8.203.84
I20260811 02:58:38.554901 9005 mini_ranger.cc:227] Ranger admin URL: http://127.8.203.84:52907
Aug 11, 2026 2:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:58:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:58:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:58:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:58:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:58:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:58:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 02:58:39.907357 9005 mini_ranger.cc:160] Time spent starting Ranger: real 8.279s user 0.002s sys 0.010s
Aug 11, 2026 2:58:59 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:59:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 2:59:00 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:59:00 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:59:01 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 02:59:01.635222 9005 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/rangerkms_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/rangerkms_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.203.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.203.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 02:59:01 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417141, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 02:59:01.718855 9005 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 02:59:01.728292 9005 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 02:59:02.440380 9005 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 02:59:02.440466 9005 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms
I20260811 02:59:02.440557 9005 mini_ranger_kms.cc:227] Using postgres at 127.8.203.84:37825
2026-08-11 02:59:02,989 [I] DB FLAVOR :POSTGRES
2026-08-11 02:59:02,989 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:59:02,989 [I] Checking connection
2026-08-11 02:59:02,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 02:59:03,305 [I] connection success
2026-08-11 02:59:03,305 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 02:59:03,305 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 02:59:03,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 02:59:03,554 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 02:59:03,554 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:59:03,554 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 02:59:03,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:37825/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 02:59:03,806 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 02:59:04.274562 9005 mini_ranger_kms.cc:310] Created kms service
I20260811 02:59:04.441558 9005 mini_ranger_kms.cc:326] Created kudu user
I20260811 02:59:04.481230 9005 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 02:59:04 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417141, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:04.866135 9005 mini_ranger_kms.cc:384] Added ranger policy
I20260811 02:59:04.866334 9005 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms:/tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 02:59:04.866359 9005 mini_ranger_kms.cc:240] Using host: 127.8.203.84
I20260811 02:59:04.870939 9005 mini_ranger_kms.cc:276] Ranger KMS PID: 31075
I20260811 02:59:04.871090 9005 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.203.84:45907
Aug 11, 2026 2:59:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:59:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:59:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 02:59:07 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786417147, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.203.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:07.561219 9005 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.842s user 0.013s sys 0.470s
I20260811 02:59:07.561349 9005 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 02:59:07.561473 9005 mini_ranger_kms.cc:401] 127.8.203.84:45907/kms/v1/keys
Aug 11 02:59:07 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417147, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.203.84@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
Aug 11 02:59:08 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417147, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.203.84@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
Aug 11 02:59:10 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:59:10.449477 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45201
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:45201
--ranger_config_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:45907/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:59:10.602243 31170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:10.602492 31170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:10.602530 31170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:10.606488 31170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:59:10.606545 31170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:10.606563 31170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:10.606577 31170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:59:10.606591 31170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:59:10.612118 31170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.203.84:45907/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:45201
--ranger_config_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:45201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:10.613154 31170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:10.614197 31170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:10.625274 31182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:10.625524 31170 server_base.cc:1034] running on GCE node
W20260811 02:59:10.625335 31183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:10.625329 31188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:10.626259 31170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:10.627311 31170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:10.628475 31170 hybrid_clock.cc:648] HybridClock initialized: now 1786417150628447 us; error 44 us; skew 500 ppm
Aug 11 02:59:10 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417150, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:10.636086 31170 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:59:10.637223 31170 webserver.cc:460] Webserver started at http://127.8.203.126:42127/ using document root <none> and password file <none>
I20260811 02:59:10.637926 31170 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:10.638024 31170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:10.638290 31170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:10 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417150, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:10.704696 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "34047cbe8f18403391ae5ec8269afd24"
format_stamp: "Formatted at 2026-08-11 02:59:10 on dist-test-slave-gjw7"
server_key: "57ac3bc51ac0869da2b97b7460ba8d5c"
server_key_iv: "73ed99686ee0090daf69eeba2a2023b7"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:10.705359 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "34047cbe8f18403391ae5ec8269afd24"
format_stamp: "Formatted at 2026-08-11 02:59:10 on dist-test-slave-gjw7"
server_key: "57ac3bc51ac0869da2b97b7460ba8d5c"
server_key_iv: "73ed99686ee0090daf69eeba2a2023b7"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:10.709479 31170 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:59:10.726109 31204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:10.727459 31170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:59:10.727588 31170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "34047cbe8f18403391ae5ec8269afd24"
format_stamp: "Formatted at 2026-08-11 02:59:10 on dist-test-slave-gjw7"
server_key: "57ac3bc51ac0869da2b97b7460ba8d5c"
server_key_iv: "73ed99686ee0090daf69eeba2a2023b7"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:10.727700 31170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:10.739692 31170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:10.742558 31170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:10.742741 31170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:10.751700 31170 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:45201
I20260811 02:59:10.751798 31311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:45201 every 8 connection(s)
I20260811 02:59:10.752758 31170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:59:10.755654 31312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:10.758219 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 31170
I20260811 02:59:10.758363 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:59:10.758674 9005 external_mini_cluster.cc:1428] Setting key 7d8611ef30eaacb78893515e4a90a776
I20260811 02:59:10.762449 31312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24: Bootstrap starting.
I20260811 02:59:10.765005 31312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:10.766470 31312 log.cc:824] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:10.768895 31312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24: No bootstrap required, opened a new log
Aug 11 02:59:10 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:10.773979 31312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34047cbe8f18403391ae5ec8269afd24" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45201 } }
I20260811 02:59:10.774317 31312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:10.774400 31312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34047cbe8f18403391ae5ec8269afd24, State: Initialized, Role: FOLLOWER
I20260811 02:59:10.774914 31312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34047cbe8f18403391ae5ec8269afd24" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45201 } }
I20260811 02:59:10.775069 31312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:59:10.775169 31312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:59:10.775275 31312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:10.775712 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:10.760249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.8.203.126:45201)
0811 02:59:10.760738 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:10.760760 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:10.760805 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:59:10.761693 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:10.761719 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:10.762162 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:10.762542 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:10.763541 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:10.765150 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:10.766337 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:10.766995 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:10.767170 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:10.770458 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:10.770468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:10.770484 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:10.770514 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:10.772424 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:10.772888 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:10.772894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:10.772899 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:10.773017 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:10.773333 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:10.773337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:10.773339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:10.773773 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:10.773992 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:10.774358 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 02:59:10.774809 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":146,"threads_started":1}
I20260811 02:59:10.776607 31312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34047cbe8f18403391ae5ec8269afd24" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45201 } }
I20260811 02:59:10.776969 31312 leader_election.cc:302] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34047cbe8f18403391ae5ec8269afd24; no voters:
I20260811 02:59:10.777369 31312 leader_election.cc:288] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:59:10.777572 31323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:59:10.778385 31323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [term 1 LEADER]: Becoming Leader. State: Replica: 34047cbe8f18403391ae5ec8269afd24, State: Running, Role: LEADER
I20260811 02:59:10.778746 31323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34047cbe8f18403391ae5ec8269afd24" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45201 } }
I20260811 02:59:10.778981 31312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:59:10.780063 31323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34047cbe8f18403391ae5ec8269afd24. Latest consensus state: current_term: 1 leader_uuid: "34047cbe8f18403391ae5ec8269afd24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34047cbe8f18403391ae5ec8269afd24" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45201 } } }
I20260811 02:59:10.780247 31323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [sys.catalog]: This master's current role is: LEADER
I20260811 02:59:10.780562 31325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34047cbe8f18403391ae5ec8269afd24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34047cbe8f18403391ae5ec8269afd24" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 45201 } } }
I20260811 02:59:10.780670 31325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24 [sys.catalog]: This master's current role is: LEADER
I20260811 02:59:10.780802 31339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:59:10.784642 31339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:59:10.785351 31312 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:59:10.797066 31339 catalog_manager.cc:1347] Generated new cluster ID: 0badf83174f94c669e395a97a19f7740
I20260811 02:59:10.797156 31339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:59:10.814389 31339 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:59:10.816833 31339 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:59:10.828722 31339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24: Generated new TSK 0
I20260811 02:59:10.829675 31339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:59:11.606385 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:45907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45201
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:59:11.777463 31404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:11.777721 31404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:11.777768 31404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:11.781912 31404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:11.781965 31404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:11.782025 31404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:59:11.787293 31404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.203.84:45907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:11.788307 31404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:11.789342 31404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:11.800057 31427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:11.799976 31419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:11.799986 31420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:11.800679 31404 server_base.cc:1034] running on GCE node
I20260811 02:59:11.801167 31404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:11.801990 31404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:11.803190 31404 hybrid_clock.cc:648] HybridClock initialized: now 1786417151803165 us; error 40 us; skew 500 ppm
Aug 11 02:59:11 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417151, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:11.806897 31404 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:59:11.807947 31404 webserver.cc:460] Webserver started at http://127.8.203.65:45105/ using document root <none> and password file <none>
I20260811 02:59:11.808630 31404 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:11.808714 31404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:11.808939 31404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:11 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417151, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:11.827138 31404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "74f5fb33eed945fd82bda2af9c013c6e"
format_stamp: "Formatted at 2026-08-11 02:59:11 on dist-test-slave-gjw7"
server_key: "55d545db6e5b363fba367ace0f506e31"
server_key_iv: "f0d346fbe8b58be9abfe4ce06d07de20"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:11.828020 31404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "74f5fb33eed945fd82bda2af9c013c6e"
format_stamp: "Formatted at 2026-08-11 02:59:11 on dist-test-slave-gjw7"
server_key: "55d545db6e5b363fba367ace0f506e31"
server_key_iv: "f0d346fbe8b58be9abfe4ce06d07de20"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:11.833495 31404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 02:59:11.866977 31449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:11.868870 31404 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.004s sys 0.000s
I20260811 02:59:11.869017 31404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "74f5fb33eed945fd82bda2af9c013c6e"
format_stamp: "Formatted at 2026-08-11 02:59:11 on dist-test-slave-gjw7"
server_key: "55d545db6e5b363fba367ace0f506e31"
server_key_iv: "f0d346fbe8b58be9abfe4ce06d07de20"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:11.869168 31404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:11.903085 31404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:11.907124 31404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:11.907398 31404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:11.908283 31404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:11.909543 31404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:11.909651 31404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:11.909781 31404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:11.909866 31404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:11.933314 31404 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:46471
I20260811 02:59:11.933475 31698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:46471 every 8 connection(s)
I20260811 02:59:11.934598 31404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:59:11.942973 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 31404
I20260811 02:59:11.943085 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:59:11.943404 9005 external_mini_cluster.cc:1428] Setting key 7fff6ff144711c15901c50e4257a441b
Aug 11 02:59:11 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417151, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:11.957578 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:11.937228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:59431 (local address 127.8.203.126:45201)
0811 02:59:11.937534 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:11.937540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:11.938795 (+ 1255us) server_negotiation.cc:407] Connection header received
0811 02:59:11.940427 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:11.940431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:11.940492 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:11.940597 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:11.942570 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:11.943402 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:11.948626 (+ 5224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:11.948993 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:11.949090 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:11.952906 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:11.952911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:11.952914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:11.952944 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:11.954899 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:11.955590 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:11.955594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:11.955595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:11.955692 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:11.956159 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:11.956164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:11.956166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:11.956422 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:11.956508 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:11.957225 (+ 717us) server_negotiation.cc:299] Negotiation successful
0811 02:59:11.957391 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 02:59:11.959334 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:11.937798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45201 (local address 127.8.203.65:59431)
0811 02:59:11.938562 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:11.938648 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:11.940157 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:11.940823 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:11.940833 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:11.941430 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:11.942356 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:11.942383 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:11.943650 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:11.943653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:11.948466 (+ 4813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:11.948479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:11.949355 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:11.949359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:11.949616 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:11.950763 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:11.950791 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:11.952702 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:11.955066 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:11.955072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:11.955090 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:11.955461 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:11.955808 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:11.955811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:11.955813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:11.956031 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:11.956586 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:11.956594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:11.956912 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 02:59:11.957858 (+ 946us) client_negotiation.cc:240] Negotiation successful
0811 02:59:11.958205 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":239,"threads_started":1}
I20260811 02:59:11.960973 31699 heartbeater.cc:347] Connected to a master server at 127.8.203.126:45201
I20260811 02:59:11.961266 31699 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 02:59:11.962056 31699 heartbeater.cc:511] Master 127.8.203.126:45201 requested a full tablet report, sending...
I20260811 02:59:11.963920 31245 ts_manager.cc:194] Registered new tserver with Master: 74f5fb33eed945fd82bda2af9c013c6e (127.8.203.65:46471)
I20260811 02:59:11.965521 31245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:59431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:59:11.984268 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:45907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45201
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 02:59:12.168273 31710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:12.168653 31710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:12.168720 31710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:12.174942 31710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:12.175027 31710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:12.175108 31710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:59:12.183032 31710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.203.84:45907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:12.184609 31710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:12.186038 31710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:12.198596 31728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:12.199007 31726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:12.199290 31710 server_base.cc:1034] running on GCE node
W20260811 02:59:12.199292 31730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:12.199988 31710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:12.200711 31710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:12.203987 31710 hybrid_clock.cc:648] HybridClock initialized: now 1786417152203948 us; error 51 us; skew 500 ppm
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:12.208420 31710 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:59:12.209650 31710 webserver.cc:460] Webserver started at http://127.8.203.66:38147/ using document root <none> and password file <none>
I20260811 02:59:12.210441 31710 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:12.210547 31710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:12.210843 31710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:12.239850 31710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "55fd818d233b417fa477a72921818b13"
format_stamp: "Formatted at 2026-08-11 02:59:12 on dist-test-slave-gjw7"
server_key: "0b0d09896499439f3e092cf0bd609a1b"
server_key_iv: "37b3d4b313e60e8f6f9920f12af1e2ab"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:12.240870 31710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "55fd818d233b417fa477a72921818b13"
format_stamp: "Formatted at 2026-08-11 02:59:12 on dist-test-slave-gjw7"
server_key: "0b0d09896499439f3e092cf0bd609a1b"
server_key_iv: "37b3d4b313e60e8f6f9920f12af1e2ab"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:12.246752 31710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:59:12.265209 31743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:12.267028 31710 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:59:12.267264 31710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "55fd818d233b417fa477a72921818b13"
format_stamp: "Formatted at 2026-08-11 02:59:12 on dist-test-slave-gjw7"
server_key: "0b0d09896499439f3e092cf0bd609a1b"
server_key_iv: "37b3d4b313e60e8f6f9920f12af1e2ab"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:12.267513 31710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:12.299998 31710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:12.302932 31710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:12.303115 31710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:12.303763 31710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:12.304759 31710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:12.304811 31710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:12.304868 31710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:12.304898 31710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:12.318353 31710 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:41565
I20260811 02:59:12.319413 31710 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:59:12.321985 31975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:41565 every 8 connection(s)
I20260811 02:59:12.325601 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 31710
I20260811 02:59:12.325724 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:59:12.326042 9005 external_mini_cluster.cc:1428] Setting key 212723a34eb369b5142306da974ab031
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:12.337502 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.323757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:39467 (local address 127.8.203.126:45201)
0811 02:59:12.323911 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.323916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.324942 (+ 1026us) server_negotiation.cc:407] Connection header received
0811 02:59:12.326466 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.326470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.326527 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.326579 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:12.328568 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.329328 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.330987 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.331260 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.331350 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.334199 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:12.334203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:12.334205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:12.334229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:12.335529 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:12.336018 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:12.336021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:12.336022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:12.336096 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:12.336411 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:12.336415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:12.336416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:12.336587 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:12.336670 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.337151 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.337278 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:59:12.338379 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.324171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45201 (local address 127.8.203.66:39467)
0811 02:59:12.324784 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:12.324839 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:12.326199 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:12.326983 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:12.326991 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:12.327427 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:12.328390 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.328414 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.329486 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.329489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.330865 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.330876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.331505 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.331508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.331714 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.332434 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:12.332454 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:12.334039 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:12.335637 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:12.335643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:12.335660 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:12.335911 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:12.336176 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:12.336179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:12.336181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:12.336324 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:12.336683 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:12.336690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:12.336925 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 02:59:12.337094 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 02:59:12.337447 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":200,"threads_started":1}
I20260811 02:59:12.339546 31977 heartbeater.cc:347] Connected to a master server at 127.8.203.126:45201
I20260811 02:59:12.339776 31977 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:12.340260 31977 heartbeater.cc:511] Master 127.8.203.126:45201 requested a full tablet report, sending...
I20260811 02:59:12.341205 31245 ts_manager.cc:194] Registered new tserver with Master: 55fd818d233b417fa477a72921818b13 (127.8.203.66:41565)
I20260811 02:59:12.341769 31245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:39467
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:59:12.399155 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:45907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45201
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
W20260811 02:59:12.589910 31984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:12.590327 31984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:12.590432 31984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:12.596691 31984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:12.596822 31984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:12.596947 31984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:59:12.604744 31984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.203.84:45907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:45201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:12.606277 31984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:12.607606 31984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:12.621431 31990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:12.621779 31984 server_base.cc:1034] running on GCE node
W20260811 02:59:12.622457 31991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:12.622869 31997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:12.623410 31984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:12.624163 31984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:12.631605 31984 hybrid_clock.cc:648] HybridClock initialized: now 1786417152631527 us; error 85 us; skew 500 ppm
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:12.635550 31984 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:59:12.636745 31984 webserver.cc:460] Webserver started at http://127.8.203.67:43455/ using document root <none> and password file <none>
I20260811 02:59:12.637672 31984 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:12.637835 31984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:12.638125 31984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:12.656471 31984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c"
format_stamp: "Formatted at 2026-08-11 02:59:12 on dist-test-slave-gjw7"
server_key: "2d9f5a621a9bd8520dbe6726a1c400aa"
server_key_iv: "758832c947b4940772b4a7e171e3c10e"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:12.657526 31984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c"
format_stamp: "Formatted at 2026-08-11 02:59:12 on dist-test-slave-gjw7"
server_key: "2d9f5a621a9bd8520dbe6726a1c400aa"
server_key_iv: "758832c947b4940772b4a7e171e3c10e"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:12.663173 31984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:59:12.682513 32011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:12.684285 31984 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:59:12.684422 31984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c"
format_stamp: "Formatted at 2026-08-11 02:59:12 on dist-test-slave-gjw7"
server_key: "2d9f5a621a9bd8520dbe6726a1c400aa"
server_key_iv: "758832c947b4940772b4a7e171e3c10e"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:12.684553 31984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:12.730474 31984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:12.735110 31984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:12.735322 31984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:12.736052 31984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:12.737115 31984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:12.737179 31984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:12.737249 31984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:12.737279 31984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:12.760579 31984 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:46121
I20260811 02:59:12.761900 31984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:59:12.765529 32224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:46121 every 8 connection(s)
I20260811 02:59:12.769685 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 31984
I20260811 02:59:12.769809 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:59:12.770097 9005 external_mini_cluster.cc:1428] Setting key 07b5704830b1f27827944d0c8bee2a80
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:12.789454 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.770481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:37697 (local address 127.8.203.126:45201)
0811 02:59:12.770858 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.770872 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.773428 (+ 2556us) server_negotiation.cc:407] Connection header received
0811 02:59:12.774940 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.774943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.774995 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.775043 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:12.777123 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.777997 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.779966 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.780327 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.780403 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.785430 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:12.785434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:12.785437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:12.785466 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:12.787140 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:12.787734 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:12.787738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:12.787740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:12.787824 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:12.788206 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:12.788210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:12.788212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:12.788421 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:12.788487 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.789149 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.789275 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280}
I20260811 02:59:12.791635 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.768289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:45201 (local address 127.8.203.67:37697)
0811 02:59:12.773255 (+ 4966us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:12.773335 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:12.774703 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:12.775470 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:12.775479 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:12.776084 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:12.776949 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.776972 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.778150 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.778153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.779839 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.779850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.780608 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.780659 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.780923 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.781743 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:12.781764 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:12.785217 (+ 3453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:12.787286 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:12.787292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:12.787309 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:12.787624 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:12.787921 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:12.787924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:12.787926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:12.788097 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:12.788568 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:12.788575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:12.788874 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 02:59:12.790176 (+ 1302us) client_negotiation.cc:240] Negotiation successful
0811 02:59:12.790547 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4700,"thread_start_us":204,"threads_started":1}
I20260811 02:59:12.793139 32225 heartbeater.cc:347] Connected to a master server at 127.8.203.126:45201
I20260811 02:59:12.793455 32225 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:12.794109 32225 heartbeater.cc:511] Master 127.8.203.126:45201 requested a full tablet report, sending...
I20260811 02:59:12.795665 31245 ts_manager.cc:194] Registered new tserver with Master: c1c4a0a68f8447f09dc18dc1319d6c6c (127.8.203.67:46121)
I20260811 02:59:12.796427 31245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:37697
I20260811 02:59:12.797766 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:59:12.810719 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.800162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.8.203.126:45201)
0811 02:59:12.800291 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.800297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.801010 (+ 713us) server_negotiation.cc:407] Connection header received
0811 02:59:12.801209 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.801213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.801261 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.801336 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:12.802434 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.803202 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.804403 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.804736 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.804812 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.806768 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:12.806773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:12.806775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:12.806802 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:12.808554 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:12.809118 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:12.809122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:12.809124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:12.809222 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:12.809607 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:12.809611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:12.809613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:12.809854 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:12.809972 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.810397 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.810527 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:59:12.814706 31245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:59:12.819276 31245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:59:12.847386 32274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.833113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:46121 (local address 127.0.0.1:35342)
0811 02:59:12.836146 (+ 3033us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:12.836158 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:12.836251 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:12.836974 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:12.836980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:12.837005 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:12.837136 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.837143 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.840474 (+ 3331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.840477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.842395 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.842407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.846865 (+ 4458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.846869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.846973 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.847014 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 02:59:12.847116 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 02:59:12.847237 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2974,"thread_start_us":767,"threads_started":1}
I20260811 02:59:12.850152 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.834566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:46471 (local address 127.0.0.1:37022)
0811 02:59:12.837381 (+ 2815us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:12.837394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:12.837476 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:12.846145 (+ 8669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:12.846148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:12.846165 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:12.846258 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.846263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.847526 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.847529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.848615 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.848625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.849807 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.849810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.849892 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.849911 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:59:12.849980 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 02:59:12.850041 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2754,"thread_start_us":128,"threads_started":1}
I20260811 02:59:12.849944 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.830856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:41565 (local address 127.0.0.1:48178)
0811 02:59:12.835735 (+ 4879us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:12.835786 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:12.835913 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:12.838675 (+ 2762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:12.838678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:12.838697 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:12.838987 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.838994 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.843773 (+ 4779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.843776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.845618 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.845627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.848816 (+ 3189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.848820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.849650 (+ 830us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.849674 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:59:12.849741 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:59:12.849809 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4701,"thread_start_us":138,"threads_started":1}
I20260811 02:59:12.850715 32269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.833585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.8.203.67:46121)
0811 02:59:12.835479 (+ 1894us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.835492 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.836337 (+ 845us) server_negotiation.cc:407] Connection header received
0811 02:59:12.836424 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.836430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.836644 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.836793 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:12.839125 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.840339 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.845768 (+ 5429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.846554 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.846652 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.846803 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.850353 (+ 3550us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.850569 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1781,"thread_start_us":147,"threads_started":1}
I20260811 02:59:12.851178 32275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.834752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.8.203.65:46471)
0811 02:59:12.845494 (+ 10742us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.845501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.845524 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:59:12.845598 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.845602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.845817 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.846004 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:12.846382 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.847404 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.848752 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.849510 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.849604 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.849750 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.850951 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.851048 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10630,"thread_start_us":91,"threads_started":1}
I20260811 02:59:12.853943 31551 tablet_service.cc:1467] Processing CreateTablet for tablet bba805bf61ac4f1dbdba5d975ad8ea56 (DEFAULT_TABLE table=test-table [id=8eda54a6a5f24e2e947673e6c88c1e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:59:12.853945 32103 tablet_service.cc:1467] Processing CreateTablet for tablet bba805bf61ac4f1dbdba5d975ad8ea56 (DEFAULT_TABLE table=test-table [id=8eda54a6a5f24e2e947673e6c88c1e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:59:12.855085 32103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba805bf61ac4f1dbdba5d975ad8ea56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:12.855086 31551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba805bf61ac4f1dbdba5d975ad8ea56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:12.863839 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.832013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.8.203.66:41565)
0811 02:59:12.838050 (+ 6037us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.838066 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.838091 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:59:12.838166 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.838171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.838378 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.838528 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:12.842564 (+ 4036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.843638 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.847729 (+ 4091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.848521 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.848618 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.848760 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.863579 (+ 14819us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.863698 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5927,"thread_start_us":132,"threads_started":1}
I20260811 02:59:12.866536 32278 tablet_bootstrap.cc:492] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Bootstrap starting.
I20260811 02:59:12.866847 31836 tablet_service.cc:1467] Processing CreateTablet for tablet bba805bf61ac4f1dbdba5d975ad8ea56 (DEFAULT_TABLE table=test-table [id=8eda54a6a5f24e2e947673e6c88c1e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:59:12.868000 31836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba805bf61ac4f1dbdba5d975ad8ea56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:12.870052 32278 tablet_bootstrap.cc:654] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:12.871398 32278 log.cc:824] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:12.872314 32276 tablet_bootstrap.cc:492] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e: Bootstrap starting.
I20260811 02:59:12.876096 32276 tablet_bootstrap.cc:654] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:12.877323 32276 log.cc:824] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:12.879582 32278 tablet_bootstrap.cc:492] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: No bootstrap required, opened a new log
I20260811 02:59:12.879783 32278 ts_tablet_manager.cc:1397] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260811 02:59:12.885700 32278 raft_consensus.cc:348] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.885986 32278 raft_consensus.cc:374] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:12.886051 32278 raft_consensus.cc:729] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c4a0a68f8447f09dc18dc1319d6c6c, State: Initialized, Role: FOLLOWER
I20260811 02:59:12.888073 32276 tablet_bootstrap.cc:492] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e: No bootstrap required, opened a new log
I20260811 02:59:12.888356 32276 ts_tablet_manager.cc:1397] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260811 02:59:12.893908 32276 raft_consensus.cc:348] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.894627 32280 tablet_bootstrap.cc:492] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Bootstrap starting.
I20260811 02:59:12.897636 32278 consensus_queue.cc:260] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.899739 32276 raft_consensus.cc:374] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:12.899852 32276 raft_consensus.cc:729] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74f5fb33eed945fd82bda2af9c013c6e, State: Initialized, Role: FOLLOWER
I20260811 02:59:12.900417 32276 consensus_queue.cc:260] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.901014 32278 ts_tablet_manager.cc:1428] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Time spent starting tablet: real 0.021s user 0.008s sys 0.000s
I20260811 02:59:12.902135 32276 ts_tablet_manager.cc:1428] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260811 02:59:12.902601 32280 tablet_bootstrap.cc:654] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:12.903043 32225 heartbeater.cc:503] Master 127.8.203.126:45201 was elected leader, sending a full tablet report...
I20260811 02:59:12.907088 31699 heartbeater.cc:503] Master 127.8.203.126:45201 was elected leader, sending a full tablet report...
I20260811 02:59:12.906961 32280 log.cc:824] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:12.912595 32280 tablet_bootstrap.cc:492] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: No bootstrap required, opened a new log
I20260811 02:59:12.912824 32280 ts_tablet_manager.cc:1397] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260811 02:59:12.919190 32280 raft_consensus.cc:348] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.919507 32280 raft_consensus.cc:374] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:12.919572 32280 raft_consensus.cc:729] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55fd818d233b417fa477a72921818b13, State: Initialized, Role: FOLLOWER
I20260811 02:59:12.920077 32280 consensus_queue.cc:260] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.921360 32280 ts_tablet_manager.cc:1428] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Time spent starting tablet: real 0.008s user 0.010s sys 0.000s
I20260811 02:59:12.921708 31977 heartbeater.cc:503] Master 127.8.203.126:45201 was elected leader, sending a full tablet report...
Aug 11 02:59:12 dist-test-slave-gjw7 krb5kdc[30263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
W20260811 02:59:12.937755 31700 tablet.cc:2369] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:59:12.977689 32287 raft_consensus.cc:482] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:59:12.977977 32287 raft_consensus.cc:504] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.979543 32287 leader_election.cc:288] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55fd818d233b417fa477a72921818b13 (127.8.203.66:41565), c1c4a0a68f8447f09dc18dc1319d6c6c (127.8.203.67:46121)
I20260811 02:59:12.988569 32269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.980689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:55165 (local address 127.8.203.67:46121)
0811 02:59:12.980838 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.980844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.981993 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 02:59:12.982172 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.982175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.982235 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.982280 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:12.982634 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.983356 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.987285 (+ 3929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.987878 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.987967 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.988014 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.988357 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.988438 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:59:12.989378 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.980481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:47165 (local address 127.8.203.66:41565)
0811 02:59:12.980862 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:12.980867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:12.980889 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:59:12.980946 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:12.980950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:12.980996 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:12.981073 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:12.981508 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.982317 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.986793 (+ 4476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.987447 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.987543 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.987600 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:12.989142 (+ 1542us) server_negotiation.cc:299] Negotiation successful
0811 02:59:12.989239 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260811 02:59:12.989573 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.979899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:46121 (local address 127.8.203.65:55165)
0811 02:59:12.981908 (+ 2009us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:12.981922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:12.982058 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:12.982408 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:12.982411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:12.982427 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:12.982521 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.982526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.985185 (+ 2659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.985188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.987136 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.987148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.988117 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.988120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.988222 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.988245 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:59:12.989230 (+ 985us) client_negotiation.cc:240] Negotiation successful
0811 02:59:12.989326 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1941,"thread_start_us":150,"threads_started":1}
I20260811 02:59:12.990294 32139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba805bf61ac4f1dbdba5d975ad8ea56" candidate_uuid: "74f5fb33eed945fd82bda2af9c013c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" is_pre_election: true
I20260811 02:59:12.990653 32139 raft_consensus.cc:2454] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74f5fb33eed945fd82bda2af9c013c6e in term 0.
I20260811 02:59:12.991211 32297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:12.979760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:41565 (local address 127.8.203.65:47165)
0811 02:59:12.980440 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:12.980457 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:12.980585 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:12.981109 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:12.981113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:12.981138 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:12.981367 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.981374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.983426 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.983429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.986658 (+ 3229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:12.986669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:12.988899 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:12.988902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:12.989007 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:12.989029 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:59:12.990856 (+ 1827us) client_negotiation.cc:240] Negotiation successful
0811 02:59:12.991060 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":120,"threads_started":1}
I20260811 02:59:12.991309 31454 leader_election.cc:302] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74f5fb33eed945fd82bda2af9c013c6e, c1c4a0a68f8447f09dc18dc1319d6c6c; no voters:
I20260811 02:59:12.991622 32287 raft_consensus.cc:2781] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:59:12.991714 32287 raft_consensus.cc:482] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:59:12.991757 32287 raft_consensus.cc:3037] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:12.992182 31880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba805bf61ac4f1dbdba5d975ad8ea56" candidate_uuid: "74f5fb33eed945fd82bda2af9c013c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55fd818d233b417fa477a72921818b13" is_pre_election: true
I20260811 02:59:12.992460 31880 raft_consensus.cc:2454] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74f5fb33eed945fd82bda2af9c013c6e in term 0.
I20260811 02:59:12.993752 32287 raft_consensus.cc:504] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:12.994083 32287 leader_election.cc:288] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [CANDIDATE]: Term 1 election: Requested vote from peers 55fd818d233b417fa477a72921818b13 (127.8.203.66:41565), c1c4a0a68f8447f09dc18dc1319d6c6c (127.8.203.67:46121)
I20260811 02:59:12.994567 32139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba805bf61ac4f1dbdba5d975ad8ea56" candidate_uuid: "74f5fb33eed945fd82bda2af9c013c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c"
I20260811 02:59:12.994596 31880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba805bf61ac4f1dbdba5d975ad8ea56" candidate_uuid: "74f5fb33eed945fd82bda2af9c013c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55fd818d233b417fa477a72921818b13"
I20260811 02:59:12.994678 32139 raft_consensus.cc:3037] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:12.994719 31880 raft_consensus.cc:3037] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:12.996343 32139 raft_consensus.cc:2454] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74f5fb33eed945fd82bda2af9c013c6e in term 1.
I20260811 02:59:12.996439 31880 raft_consensus.cc:2454] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74f5fb33eed945fd82bda2af9c013c6e in term 1.
I20260811 02:59:12.996737 31454 leader_election.cc:302] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74f5fb33eed945fd82bda2af9c013c6e, c1c4a0a68f8447f09dc18dc1319d6c6c; no voters:
I20260811 02:59:12.996948 32287 raft_consensus.cc:2781] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:59:12.997152 32287 raft_consensus.cc:686] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [term 1 LEADER]: Becoming Leader. State: Replica: 74f5fb33eed945fd82bda2af9c013c6e, State: Running, Role: LEADER
I20260811 02:59:12.997530 32287 consensus_queue.cc:237] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } }
I20260811 02:59:13.001152 31245 catalog_manager.cc:5495] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 74f5fb33eed945fd82bda2af9c013c6e (127.8.203.65). New cstate: current_term: 1 leader_uuid: "74f5fb33eed945fd82bda2af9c013c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74f5fb33eed945fd82bda2af9c013c6e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 46471 } health_report { overall_health: HEALTHY } } }
W20260811 02:59:13.025637 32232 tablet.cc:2369] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:59:13.045547 32275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:13.040042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.8.203.65:46471)
0811 02:59:13.040196 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:13.040202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:13.040221 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:59:13.040386 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:13.040390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:13.040461 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:13.040554 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:59:13.040961 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:13.041797 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:13.043017 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:13.043358 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:13.043463 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:13.043908 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:59:13.044525 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:59:13.044646 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:13.045170 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 02:59:13.045246 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:59:13.053843 32139 raft_consensus.cc:1261] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 1 FOLLOWER]: Refusing update from remote peer 74f5fb33eed945fd82bda2af9c013c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:59:13.054836 32287 consensus_queue.cc:1035] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1c4a0a68f8447f09dc18dc1319d6c6c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 46121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:59:13.061827 31880 raft_consensus.cc:1261] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 1 FOLLOWER]: Refusing update from remote peer 74f5fb33eed945fd82bda2af9c013c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:59:13.063104 32287 consensus_queue.cc:1035] T bba805bf61ac4f1dbdba5d975ad8ea56 P 74f5fb33eed945fd82bda2af9c013c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "55fd818d233b417fa477a72921818b13" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:59:13.074059 31978 tablet.cc:2369] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:59:13.078698 32309 mvcc.cc:205] Tried to move back new op lower bound from 7317164658898325504 to 7317164658681380864. Current Snapshot: MvccSnapshot[applied={T|T < 7317164658898325504}]
I20260811 02:59:13.081027 32320 mvcc.cc:205] Tried to move back new op lower bound from 7317164658898325504 to 7317164658681380864. Current Snapshot: MvccSnapshot[applied={T|T < 7317164658898325504}]
I20260811 02:59:13.093367 32315 mvcc.cc:205] Tried to move back new op lower bound from 7317164658898325504 to 7317164658681380864. Current Snapshot: MvccSnapshot[applied={T|T < 7317164658898325504}]
I20260811 02:59:13.102547 31245 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:59:13.102768 31245 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:59:13.106043 31245 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34047cbe8f18403391ae5ec8269afd24: Sending DeleteTablet for 3 replicas of tablet bba805bf61ac4f1dbdba5d975ad8ea56
I20260811 02:59:13.106901 31836 tablet_service.cc:1514] Processing DeleteTablet for tablet bba805bf61ac4f1dbdba5d975ad8ea56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:59:13 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:48178
I20260811 02:59:13.107347 32103 tablet_service.cc:1514] Processing DeleteTablet for tablet bba805bf61ac4f1dbdba5d975ad8ea56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:59:13 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:35342
I20260811 02:59:13.108197 31551 tablet_service.cc:1514] Processing DeleteTablet for tablet bba805bf61ac4f1dbdba5d975ad8ea56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:59:13 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:37022
I20260811 02:59:13.109400 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 31404
I20260811 02:59:13.129563 32334 tablet_replica.cc:330] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: stopping tablet replica
I20260811 02:59:13.129935 32334 raft_consensus.cc:2229] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:59:13.130270 32334 raft_consensus.cc:2258] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:59:13.135068 32334 ts_tablet_manager.cc:1905] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:59:13.138072 32338 tablet_replica.cc:330] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: stopping tablet replica
I20260811 02:59:13.138739 32338 raft_consensus.cc:2229] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:59:13.142319 32338 raft_consensus.cc:2258] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:59:13.146864 32338 ts_tablet_manager.cc:1905] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:59:13.149957 32338 ts_tablet_manager.cc:1918] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:59:13.150174 32338 log.cc:1196] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/bba805bf61ac4f1dbdba5d975ad8ea56
I20260811 02:59:13.150369 32334 ts_tablet_manager.cc:1918] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:59:13.150609 32338 ts_tablet_manager.cc:1939] T bba805bf61ac4f1dbdba5d975ad8ea56 P c1c4a0a68f8447f09dc18dc1319d6c6c: Deleting consensus metadata
I20260811 02:59:13.150619 32334 log.cc:1196] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/bba805bf61ac4f1dbdba5d975ad8ea56
I20260811 02:59:13.150974 32334 ts_tablet_manager.cc:1939] T bba805bf61ac4f1dbdba5d975ad8ea56 P 55fd818d233b417fa477a72921818b13: Deleting consensus metadata
I20260811 02:59:13.152267 31211 catalog_manager.cc:4841] TS 55fd818d233b417fa477a72921818b13 (127.8.203.66:41565): tablet bba805bf61ac4f1dbdba5d975ad8ea56 (table test-table [id=8eda54a6a5f24e2e947673e6c88c1e04]) successfully deleted
I20260811 02:59:13.153414 31212 catalog_manager.cc:4841] TS c1c4a0a68f8447f09dc18dc1319d6c6c (127.8.203.67:46121): tablet bba805bf61ac4f1dbdba5d975ad8ea56 (table test-table [id=8eda54a6a5f24e2e947673e6c88c1e04]) successfully deleted
W20260811 02:59:13.154477 31209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:46471 (error 108)
I20260811 02:59:13.155390 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 31710
I20260811 02:59:13.156144 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:13.155299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:46471 (local address 127.0.0.1:37032)
0811 02:59:13.155956 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:13.156044 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514}
W20260811 02:59:13.158576 31209 catalog_manager.cc:4584] TS 74f5fb33eed945fd82bda2af9c013c6e (127.8.203.65:46471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bba805bf61ac4f1dbdba5d975ad8ea56: Network error: Client connection negotiation failed: client connection to 127.8.203.65:46471: connect: Connection refused (error 111)
I20260811 02:59:13.215019 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:13.211462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:46471 (local address 127.0.0.1:37038)
0811 02:59:13.214795 (+ 3333us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:13.214890 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3203}
I20260811 02:59:13.216190 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:13.215672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:46471 (local address 127.0.0.1:37052)
0811 02:59:13.216069 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:13.216122 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:46471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321}
I20260811 02:59:13.224000 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 31984
I20260811 02:59:13.234467 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 31170
I20260811 02:59:13.253099 9005 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 02:59:13.596686 9005 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 02:59:13.596818 9005 mini_ranger.cc:66] Stopping Ranger...
I20260811 02:59:13.940547 9005 mini_ranger.cc:68] Stopped Ranger
2026-08-11 02:59:13.940 UTC [30341] LOG: received smart shutdown request
2026-08-11 02:59:13.944 UTC [30341] LOG: background worker "logical replication launcher" (PID 30356) exited with exit code 1
2026-08-11 02:59:13.945 UTC [30347] LOG: shutting down
2026-08-11 02:59:14.076 UTC [30341] LOG: database system is shut down
2026-08-11T02:59:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47214 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:59:14 dist-test-slave-gjw7 krb5kdc[32364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:59:14 dist-test-slave-gjw7 krb5kdc[32364](info): set up 2 sockets
Aug 11 02:59:14 dist-test-slave-gjw7 krb5kdc[32364](info): commencing operation
krb5kdc: starting...
W20260811 02:59:15.364784 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 02:59:15 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:59:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/rangeradmin_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/rangeradmin_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/rangerlookup_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/rangerlookup_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
I20260811 02:59:15.520780 9005 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1jpVEs/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/postgres -l logfile start
2026-08-11 02:59:17.629 UTC [32427] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 02:59:17.629 UTC [32427] LOG: listening on IPv4 address "127.8.203.84", port 58907
2026-08-11 02:59:17.632 UTC [32427] LOG: listening on Unix socket "/tmp/.s.PGSQL.58907"
2026-08-11 02:59:17.652 UTC [32434] LOG: database system was shut down at 2026-08-11 02:59:16 UTC
2026-08-11 02:59:17.655 UTC [32427] LOG: database system is ready to accept connections
I20260811 02:59:18.766503 9005 mini_postgres.cc:96] Postgres bound to 58907
2026-08-11 02:59:18.773 UTC [32446] FATAL: database "slave" does not exist
127.8.203.84:58907 - accepting connections
I20260811 02:59:18.774431 9005 mini_ranger.cc:161] Starting Ranger...
I20260811 02:59:18.789741 9005 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 02:59:19.171638 9005 mini_ranger.cc:87] Created ranger Postgres database
I20260811 02:59:19.171725 9005 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-admin
2026-08-11 02:59:19,432 [I] DB FLAVOR :POSTGRES
2026-08-11 02:59:19,432 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:59:19,432 [I] Checking connection..
2026-08-11 02:59:19,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 02:59:19,672 [I] Checking connection passed.
2026-08-11 02:59:19,672 [I] --------- Verifying version history table ---------
2026-08-11 02:59:19,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:59:19,912 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:59:19,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:59:20,139 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:59:20,139 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 02:59:20,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 02:59:20.393 UTC [32576] WARNING: there is no transaction in progress
2026-08-11 02:59:20,401 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 02:59:20,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:59:20,660 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 02:59:20,660 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:59:20,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 02:59:20,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 02:59:21,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-11 02:59:21,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:59:21,621 [I] Table x_portal_user does not exist in database ranger
2026-08-11 02:59:21,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:59:21,852 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 02:59:21,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:59:22,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:59:22,349 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 02:59:22,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 02:59:23.226 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.240 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.254 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.266 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.280 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.296 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.345 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.351 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.364 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.373 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.383 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.395 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.403 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.409 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.414 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23.420 UTC [325] WARNING: there is no transaction in progress
2026-08-11 02:59:23,955 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 02:59:23,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 02:59:24,203 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 02:59:24,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:59:24,440 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 02:59:24,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:59:24,670 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 02:59:24,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:59:24,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:59:25,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-11 02:59:25,422 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 02:59:25,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 02:59:25,675 [I] DB_PATCHES have already been applied
I20260811 02:59:25.681466 9005 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-admin:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1jpVEs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 02:59:25.681537 9005 mini_ranger.cc:193] Using host: 127.8.203.84
I20260811 02:59:25.685782 9005 mini_ranger.cc:227] Ranger admin URL: http://127.8.203.84:36341
Aug 11, 2026 2:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:59:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:59:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 02:59:27.016180 9005 mini_ranger.cc:160] Time spent starting Ranger: real 8.242s user 0.000s sys 0.010s
Aug 11, 2026 2:59:46 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:59:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 2:59:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:59:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:59:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 02:59:47.670420 9005 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.203.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.203.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/rangerkms_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.203.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/rangerkms_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.203.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.203.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.203.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/HTTP_127.8.203.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 02:59:47 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417187, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 02:59:47.749570 9005 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 02:59:47.758409 9005 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 02:59:48.456626 9005 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 02:59:48.456709 9005 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms
I20260811 02:59:48.456725 9005 mini_ranger_kms.cc:227] Using postgres at 127.8.203.84:58907
2026-08-11 02:59:48,610 [I] DB FLAVOR :POSTGRES
2026-08-11 02:59:48,610 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:59:48,610 [I] Checking connection
2026-08-11 02:59:48,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 02:59:48,841 [I] connection success
2026-08-11 02:59:48,841 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 02:59:48,841 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 02:59:48,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 02:59:49,076 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 02:59:49,077 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:59:49,077 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 02:59:49,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.203.84:58907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 02:59:49,324 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 02:59:49.739624 9005 mini_ranger_kms.cc:310] Created kms service
I20260811 02:59:49.931169 9005 mini_ranger_kms.cc:326] Created kudu user
I20260811 02:59:49.973058 9005 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 02:59:49 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417187, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:50.357672 9005 mini_ranger_kms.cc:384] Added ranger policy
I20260811 02:59:50.357836 9005 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms:/tmp/dist-test-task1jpVEs/build/debug/bin/postgresql.jar:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1jpVEs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 02:59:50.357869 9005 mini_ranger_kms.cc:240] Using host: 127.8.203.84
I20260811 02:59:50.362596 9005 mini_ranger_kms.cc:276] Ranger KMS PID: 724
I20260811 02:59:50.362679 9005 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.203.84:43473
Aug 11, 2026 2:59:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:59:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:59:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:59:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:59:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:59:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 02:59:52 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786417192, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.203.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:52.958973 9005 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.209s user 0.020s sys 0.120s
I20260811 02:59:52.959095 9005 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 02:59:52.959161 9005 mini_ranger_kms.cc:401] 127.8.203.84:43473/kms/v1/keys
Aug 11 02:59:53 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417192, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.203.84@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
Aug 11 02:59:53 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417192, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.203.84@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
Aug 11 02:59:55 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 02:59:55.526007 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:42095
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:42095
--ranger_config_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:59:55.634123 824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:55.634351 824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:55.634393 824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:55.638242 824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:59:55.638294 824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:55.638311 824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:55.638330 824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:59:55.638345 824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:59:55.643535 824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.203.84:43473/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:42095
--ranger_config_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:42095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:55.644430 824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:55.645349 824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:55.653308 843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:55.653334 835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:55.653355 834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:55.653525 824 server_base.cc:1034] running on GCE node
I20260811 02:59:55.654043 824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:55.655057 824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:55.656209 824 hybrid_clock.cc:648] HybridClock initialized: now 1786417195656197 us; error 25 us; skew 500 ppm
Aug 11 02:59:55 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417195, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:55.661479 824 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:59:55.662307 824 webserver.cc:460] Webserver started at http://127.8.203.126:38481/ using document root <none> and password file <none>
I20260811 02:59:55.662814 824 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:55.662868 824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:55.663065 824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:55 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417195, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:55.716555 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "2af679b973704ff4962e294e468e4567"
format_stamp: "Formatted at 2026-08-11 02:59:55 on dist-test-slave-gjw7"
server_key: "038b6fdafd1b7d2fbd0b4d020366952c"
server_key_iv: "0fe1901751fdb27408a7d7b9dc2e252d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:55.717373 824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "2af679b973704ff4962e294e468e4567"
format_stamp: "Formatted at 2026-08-11 02:59:55 on dist-test-slave-gjw7"
server_key: "038b6fdafd1b7d2fbd0b4d020366952c"
server_key_iv: "0fe1901751fdb27408a7d7b9dc2e252d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:55.722514 824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:59:55.739277 852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:55.740935 824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:59:55.741079 824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "2af679b973704ff4962e294e468e4567"
format_stamp: "Formatted at 2026-08-11 02:59:55 on dist-test-slave-gjw7"
server_key: "038b6fdafd1b7d2fbd0b4d020366952c"
server_key_iv: "0fe1901751fdb27408a7d7b9dc2e252d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:55.741222 824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:55.774735 824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:55.777731 824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:55.777922 824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:55.784933 824 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:42095
I20260811 02:59:55.784981 959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:42095 every 8 connection(s)
I20260811 02:59:55.785916 824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:59:55.789465 963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:55.792793 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 824
I20260811 02:59:55.792893 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 02:59:55.793159 9005 external_mini_cluster.cc:1428] Setting key 29a145f0d731570597216728294cbf06
I20260811 02:59:55.796852 963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Bootstrap starting.
I20260811 02:59:55.799691 963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:55.800724 963 log.cc:824] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:59:55 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:55.803190 963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: No bootstrap required, opened a new log
I20260811 02:59:55.807246 971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:55.794620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.8.203.126:42095)
0811 02:59:55.795043 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:55.795058 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:55.795095 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:59:55.795768 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:55.795787 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:55.796198 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:55.796524 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:55.797374 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:55.798311 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:55.799162 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:55.799623 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:55.799770 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:55.802310 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:55.802316 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:55.802330 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:55.802357 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:55.804482 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:55.804888 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:55.804893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:55.804898 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:55.805004 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:55.805257 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:55.805260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:55.805262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:55.805719 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:55.805893 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:55.806183 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 02:59:55.806488 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":111,"threads_started":1}
I20260811 02:59:55.807389 963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:55.807683 963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:55.807777 963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af679b973704ff4962e294e468e4567, State: Initialized, Role: FOLLOWER
I20260811 02:59:55.808270 963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:55.808533 963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:59:55.808610 963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:59:55.808725 963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:55.809923 963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:55.810282 963 leader_election.cc:302] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2af679b973704ff4962e294e468e4567; no voters:
I20260811 02:59:55.810618 963 leader_election.cc:288] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:59:55.810719 973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:59:55.810936 973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 LEADER]: Becoming Leader. State: Replica: 2af679b973704ff4962e294e468e4567, State: Running, Role: LEADER
I20260811 02:59:55.811275 973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:55.812047 963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:59:55.813171 974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2af679b973704ff4962e294e468e4567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } } }
I20260811 02:59:55.813304 974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: This master's current role is: LEADER
I20260811 02:59:55.813699 975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2af679b973704ff4962e294e468e4567. Latest consensus state: current_term: 1 leader_uuid: "2af679b973704ff4962e294e468e4567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } } }
I20260811 02:59:55.813850 975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: This master's current role is: LEADER
I20260811 02:59:55.814502 993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:59:55.814612 963 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:59:55.818782 993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:59:55.829435 993 catalog_manager.cc:1347] Generated new cluster ID: ffdfb7055e17414fbdb1d55714d5fec0
I20260811 02:59:55.829517 993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:59:55.858055 993 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:59:55.859524 993 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:59:55.867956 993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Generated new TSK 0
I20260811 02:59:55.868958 993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 02:59:56.554600 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42095
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:59:56.718634 1053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:56.718868 1053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:56.718924 1053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:56.722875 1053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:56.722929 1053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:56.723006 1053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:59:56.728109 1053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.203.84:43473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:56.729103 1053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:56.730111 1053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:56.740954 1068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:56.741113 1053 server_base.cc:1034] running on GCE node
W20260811 02:59:56.741178 1065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:56.741367 1071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:56.741741 1053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:56.742322 1053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:56.743497 1053 hybrid_clock.cc:648] HybridClock initialized: now 1786417196743475 us; error 32 us; skew 500 ppm
Aug 11 02:59:56 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:56.746116 1053 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:59:56.747017 1053 webserver.cc:460] Webserver started at http://127.8.203.65:35095/ using document root <none> and password file <none>
I20260811 02:59:56.747553 1053 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:56.747618 1053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:56.747846 1053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:56 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:56.769141 1053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "2861acdd94f448ad8514a9bab7e41433"
format_stamp: "Formatted at 2026-08-11 02:59:56 on dist-test-slave-gjw7"
server_key: "e45ab53b554ed9bafdc50c59b5531f4b"
server_key_iv: "b20415bc87c20eb761b5587febc4e89d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:56.769924 1053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "2861acdd94f448ad8514a9bab7e41433"
format_stamp: "Formatted at 2026-08-11 02:59:56 on dist-test-slave-gjw7"
server_key: "e45ab53b554ed9bafdc50c59b5531f4b"
server_key_iv: "b20415bc87c20eb761b5587febc4e89d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:56.775147 1053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:59:56.797642 1097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:56.799345 1053 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260811 02:59:56.799494 1053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "2861acdd94f448ad8514a9bab7e41433"
format_stamp: "Formatted at 2026-08-11 02:59:56 on dist-test-slave-gjw7"
server_key: "e45ab53b554ed9bafdc50c59b5531f4b"
server_key_iv: "b20415bc87c20eb761b5587febc4e89d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:56.799641 1053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:56.833634 1053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:56.837278 1053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:56.837540 1053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:56.838256 1053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:56.839397 1053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:56.839504 1053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:56.839628 1053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:56.839718 1053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:56.850679 1053 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:44761
I20260811 02:59:56.851882 1053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:59:56.853488 1342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:44761 every 8 connection(s)
I20260811 02:59:56.858820 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 1053
I20260811 02:59:56.858943 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 02:59:56.859244 9005 external_mini_cluster.cc:1428] Setting key ce709f117f64f390d7ef26739f793561
Aug 11 02:59:56 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 02:59:56.875268 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:56.855980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:45675 (local address 127.8.203.126:42095)
0811 02:59:56.861471 (+ 5491us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:56.861481 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:56.863156 (+ 1675us) server_negotiation.cc:407] Connection header received
0811 02:59:56.863219 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:56.863222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:56.863279 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:56.863372 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:56.864843 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:56.865766 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:56.867483 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:56.867850 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:56.867925 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:56.870948 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:56.870952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:56.870955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:56.870980 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:56.872769 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:56.873399 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:56.873403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:56.873404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:56.873492 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:56.873898 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:56.873901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:56.873903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:56.874125 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:56.874264 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:56.874940 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 02:59:56.875073 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5402,"thread_start_us":95,"threads_started":1}
I20260811 02:59:56.876868 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:56.856473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42095 (local address 127.8.203.65:45675)
0811 02:59:56.861765 (+ 5292us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:56.861823 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:56.862856 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:56.863605 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:56.863615 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:56.863984 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:56.864666 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:56.864686 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:56.865936 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:56.865939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:56.867343 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:56.867355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:56.868131 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:56.868135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:56.868341 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:56.869003 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:56.869024 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:56.870762 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:56.872906 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:56.872912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:56.872927 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:56.873255 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:56.873597 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:56.873600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:56.873602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:56.873784 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:56.874341 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:56.874347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:56.874654 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 02:59:56.875503 (+ 849us) client_negotiation.cc:240] Negotiation successful
0811 02:59:56.875840 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5056,"thread_start_us":151,"threads_started":1}
I20260811 02:59:56.878477 1344 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42095
I20260811 02:59:56.878721 1344 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:56.879345 1344 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:56.881108 883 ts_manager.cc:194] Registered new tserver with Master: 2861acdd94f448ad8514a9bab7e41433 (127.8.203.65:44761)
I20260811 02:59:56.882702 883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:45675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 02:59:56.892555 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42095
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:59:56 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786417196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 02:59:57.086484 1362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:57.086818 1362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:57.086910 1362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:57.092700 1362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:57.092782 1362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:57.092880 1362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:59:57.100917 1362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.203.84:43473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:57.102342 1362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:57.103709 1362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:57.118506 1374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:57.118530 1372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:57.119199 1371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:57.120848 1362 server_base.cc:1034] running on GCE node
I20260811 02:59:57.121373 1362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:57.122231 1362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:57.123430 1362 hybrid_clock.cc:648] HybridClock initialized: now 1786417197123405 us; error 40 us; skew 500 ppm
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:57.127888 1362 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:59:57.129101 1362 webserver.cc:460] Webserver started at http://127.8.203.66:36665/ using document root <none> and password file <none>
I20260811 02:59:57.129897 1362 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:57.130003 1362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:57.130286 1362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:57.158004 1362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "54fe9a68171b4f8ca9c611147bd773e8"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "33e5f5755221a5e6baf9e09249d669e2"
server_key_iv: "4d1e4031b921913c38e43f63ca3ce9cf"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:57.158955 1362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "54fe9a68171b4f8ca9c611147bd773e8"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "33e5f5755221a5e6baf9e09249d669e2"
server_key_iv: "4d1e4031b921913c38e43f63ca3ce9cf"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:57.164655 1362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 02:59:57.209504 1385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:57.211378 1362 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.001s
I20260811 02:59:57.211560 1362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "54fe9a68171b4f8ca9c611147bd773e8"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "33e5f5755221a5e6baf9e09249d669e2"
server_key_iv: "4d1e4031b921913c38e43f63ca3ce9cf"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:57.211748 1362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:57.234906 1362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:57.237775 1362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:57.237960 1362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:57.238510 1362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:57.239370 1362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:57.239419 1362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:57.239471 1362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:57.239497 1362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:57.254570 1362 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:37163
I20260811 02:59:57.254838 1621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:37163 every 8 connection(s)
I20260811 02:59:57.255578 1362 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 02:59:57.262692 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 1362
I20260811 02:59:57.262813 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 02:59:57.263105 9005 external_mini_cluster.cc:1428] Setting key 19cfdf5f780b8fcc90d3cab863fc43c8
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:57.272015 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:57.257479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:32781 (local address 127.8.203.126:42095)
0811 02:59:57.257653 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:57.257659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:57.258594 (+ 935us) server_negotiation.cc:407] Connection header received
0811 02:59:57.259574 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:57.259577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:57.259635 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:57.259709 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:57.260864 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.261439 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.264046 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.264412 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.264497 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:57.267632 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:57.267636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:57.267639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:57.267669 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:57.269471 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:57.270110 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:57.270114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:57.270116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:57.270209 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:57.270652 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:57.270655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:57.270657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:57.270889 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:57.270964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:57.271697 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 02:59:57.271833 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:59:57.273660 1626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:57.257914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42095 (local address 127.8.203.66:32781)
0811 02:59:57.258442 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:57.258496 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:57.259367 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:57.259898 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:57.259905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:57.260217 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:57.260712 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:57.260727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.261553 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.261557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:57.263905 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:57.263918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.264704 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.264708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:57.264930 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:57.265660 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:57.265683 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:57.267407 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:57.269620 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:57.269626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:57.269643 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:57.269989 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:57.270329 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:57.270332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:57.270346 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:57.270531 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:57.271048 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:57.271055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:57.271388 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 02:59:57.272187 (+ 799us) client_negotiation.cc:240] Negotiation successful
0811 02:59:57.272534 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":120,"threads_started":1}
I20260811 02:59:57.275123 1622 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42095
I20260811 02:59:57.275367 1622 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:57.276005 1622 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:57.277213 883 ts_manager.cc:194] Registered new tserver with Master: 54fe9a68171b4f8ca9c611147bd773e8 (127.8.203.66:37163)
I20260811 02:59:57.277977 883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:32781
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 02:59:57.322603 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42095
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
W20260811 02:59:57.552388 1631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:57.552727 1631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:57.552795 1631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:57.559006 1631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:57.559082 1631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:57.559170 1631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:59:57.568662 1631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.203.84:43473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:57.570010 1631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:57.571290 1631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:57.585458 1642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:57.585458 1641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:57.585736 1645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:57.587090 1631 server_base.cc:1034] running on GCE node
I20260811 02:59:57.587630 1631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:57.588409 1631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:57.603875 1631 hybrid_clock.cc:648] HybridClock initialized: now 1786417197603848 us; error 50 us; skew 500 ppm
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:57.608649 1631 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:59:57.609823 1631 webserver.cc:460] Webserver started at http://127.8.203.67:44507/ using document root <none> and password file <none>
I20260811 02:59:57.610554 1631 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:57.610651 1631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:59:57.610903 1631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:57.633455 1631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "eeb1b373865041598f22148dbad7e85a"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "cb22fc4d8b4a3224aa98a959be6616dd"
server_key_iv: "38a52ce3992009147c9c2bdd177a4fe4"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:57.634289 1631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "eeb1b373865041598f22148dbad7e85a"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "cb22fc4d8b4a3224aa98a959be6616dd"
server_key_iv: "38a52ce3992009147c9c2bdd177a4fe4"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:57.647089 1631 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.005s sys 0.000s
I20260811 02:59:57.673213 1659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:57.675155 1631 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260811 02:59:57.675316 1631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "eeb1b373865041598f22148dbad7e85a"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "cb22fc4d8b4a3224aa98a959be6616dd"
server_key_iv: "38a52ce3992009147c9c2bdd177a4fe4"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:57.675498 1631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:57.696054 1631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:57.699662 1631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:57.699910 1631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:57.700640 1631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:57.701772 1631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:57.701833 1631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:57.701890 1631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:57.701921 1631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:57.718240 1631 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:34915
I20260811 02:59:57.719306 1631 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:59:57.720099 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 1631
I20260811 02:59:57.720204 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 02:59:57.720482 9005 external_mini_cluster.cc:1428] Setting key e108d667a160180e80b28373944c3cf7
I20260811 02:59:57.729887 1886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:34915 every 8 connection(s)
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:57.736580 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:57.721374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:57987 (local address 127.8.203.126:42095)
0811 02:59:57.721497 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:57.721502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:57.723419 (+ 1917us) server_negotiation.cc:407] Connection header received
0811 02:59:57.724634 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:57.724637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:57.724692 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:57.724782 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:57.726291 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.727126 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.728800 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.729138 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.729241 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:57.732161 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:57.732175 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:57.732179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:57.732211 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:57.734093 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:57.734791 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:57.734794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:57.734796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:57.734902 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:57.735320 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:57.735324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:57.735325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:57.735552 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:57.735648 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:57.736301 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 02:59:57.736425 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:59:57.737875 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:57.721949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42095 (local address 127.8.203.67:57987)
0811 02:59:57.723237 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:57.723296 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:57.724387 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:57.724976 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:57.724987 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:57.725410 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:57.726110 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:57.726130 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.727274 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.727278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:57.728651 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:57.728664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:57.729321 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:57.729327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:57.729530 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:57.730222 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:57.730245 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:57.731969 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:57.734262 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:57.734269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:57.734285 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:57.734651 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:57.735000 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:57.735003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:57.735005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:57.735206 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:57.735657 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:57.735664 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:57.735998 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 02:59:57.736218 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 02:59:57.736549 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"thread_start_us":179,"threads_started":1}
I20260811 02:59:57.739105 1887 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42095
I20260811 02:59:57.739374 1887 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:57.739954 1887 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:57.741153 883 ts_manager.cc:194] Registered new tserver with Master: eeb1b373865041598f22148dbad7e85a (127.8.203.67:34915)
I20260811 02:59:57.742053 883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:57987
I20260811 02:59:57.746455 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:59:57.747336 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 1053
I20260811 02:59:57.768528 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 1362
I20260811 02:59:57.779901 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 1631
I20260811 02:59:57.795455 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 824
I20260811 02:59:57.827162 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:42095
--webserver_interface=127.8.203.126
--webserver_port=38481
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:42095
--ranger_config_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:59:57.939427 1893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:57.939678 1893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:57.939721 1893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:57.943661 1893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:59:57.943710 1893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:57.943730 1893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:57.943760 1893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:59:57.943787 1893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:59:57.948993 1893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:42095
--ranger_config_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:42095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=38481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:57.949965 1893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:57.950924 1893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:57.959163 1914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:57.959335 1893 server_base.cc:1034] running on GCE node
W20260811 02:59:57.959194 1908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:57.959194 1905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:57.959990 1893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:57.960978 1893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:57.962137 1893 hybrid_clock.cc:648] HybridClock initialized: now 1786417197962118 us; error 30 us; skew 500 ppm
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:57.964674 1893 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 02:59:57.965547 1893 webserver.cc:460] Webserver started at http://127.8.203.126:38481/ using document root <none> and password file <none>
I20260811 02:59:57.966051 1893 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:57.966125 1893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:59:57 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:58.004177 1893 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:59:58.006852 1926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.008118 1893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:59:58.008231 1893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "2af679b973704ff4962e294e468e4567"
format_stamp: "Formatted at 2026-08-11 02:59:55 on dist-test-slave-gjw7"
server_key: "038b6fdafd1b7d2fbd0b4d020366952c"
server_key_iv: "0fe1901751fdb27408a7d7b9dc2e252d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:58.008783 1893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:58.020323 1893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:58.022645 1893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:58.022807 1893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:58.029511 1893 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:42095
I20260811 02:59:58.029538 2021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:42095 every 8 connection(s)
I20260811 02:59:58.030504 1893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 02:59:58.033792 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 1893
I20260811 02:59:58.034274 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:44761
--local_ip_for_outbound_sockets=127.8.203.65
--tserver_master_addrs=127.8.203.126:42095
--webserver_port=35095
--webserver_interface=127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:59:58.034857 2022 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:59:58.036749 2022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Bootstrap starting.
I20260811 02:59:58.052608 2022 log.cc:824] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:58.058950 2022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:59:58.059286 2022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Bootstrap complete.
I20260811 02:59:58.063180 2022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:58.063561 2022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af679b973704ff4962e294e468e4567, State: Initialized, Role: FOLLOWER
I20260811 02:59:58.064047 2022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:58.064163 2022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:59:58.064224 2022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:59:58.064317 2022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:59:58.066162 2022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:58.066473 2022 leader_election.cc:302] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2af679b973704ff4962e294e468e4567; no voters:
I20260811 02:59:58.066743 2022 leader_election.cc:288] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:59:58.066903 2031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:59:58.067157 2031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [term 2 LEADER]: Becoming Leader. State: Replica: 2af679b973704ff4962e294e468e4567, State: Running, Role: LEADER
I20260811 02:59:58.067466 2031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } }
I20260811 02:59:58.067876 2022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:59:58.068943 2035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2af679b973704ff4962e294e468e4567. Latest consensus state: current_term: 2 leader_uuid: "2af679b973704ff4962e294e468e4567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } } }
I20260811 02:59:58.068969 2033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2af679b973704ff4962e294e468e4567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2af679b973704ff4962e294e468e4567" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42095 } } }
I20260811 02:59:58.069063 2035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: This master's current role is: LEADER
I20260811 02:59:58.069070 2033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567 [sys.catalog]: This master's current role is: LEADER
I20260811 02:59:58.069545 2053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:59:58.070060 2022 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:59:58.076385 2053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:59:58.077062 2053 catalog_manager.cc:1259] Loaded cluster ID: ffdfb7055e17414fbdb1d55714d5fec0
I20260811 02:59:58.077123 2053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:59:58.079828 2053 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:59:58.080551 2053 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Loaded TSK: 0
I20260811 02:59:58.081926 2053 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:59:58.177043 2026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:58.177397 2026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:58.177466 2026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:58.183228 2026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:58.183292 2026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:58.183372 2026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 02:59:58.190724 2026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:44761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=35095
--tserver_master_addrs=127.8.203.126:42095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:58.191960 2026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:58.193253 2026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:58.205693 2095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:58.206063 2026 server_base.cc:1034] running on GCE node
W20260811 02:59:58.206293 2106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:58.206669 2094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:58.207202 2026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:58.208127 2026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:58.209347 2026 hybrid_clock.cc:648] HybridClock initialized: now 1786417198209319 us; error 39 us; skew 500 ppm
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:58.217592 2026 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 02:59:58.218709 2026 webserver.cc:460] Webserver started at http://127.8.203.65:35095/ using document root <none> and password file <none>
I20260811 02:59:58.219435 2026 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:58.219543 2026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:58.262878 2026 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.003s sys 0.000s
I20260811 02:59:58.271250 2117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.272737 2026 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260811 02:59:58.272874 2026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "2861acdd94f448ad8514a9bab7e41433"
format_stamp: "Formatted at 2026-08-11 02:59:56 on dist-test-slave-gjw7"
server_key: "e45ab53b554ed9bafdc50c59b5531f4b"
server_key_iv: "b20415bc87c20eb761b5587febc4e89d"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:58.273538 2026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:58.292934 2026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:58.296026 2026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:58.296198 2026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:58.296797 2026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:58.297849 2026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:58.297911 2026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.297964 2026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:58.297992 2026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.308338 2026 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:44761
I20260811 02:59:58.308929 2347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:44761 every 8 connection(s)
I20260811 02:59:58.309468 2026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 02:59:58.316849 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 2026
I20260811 02:59:58.317381 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:37163
--local_ip_for_outbound_sockets=127.8.203.66
--tserver_master_addrs=127.8.203.126:42095
--webserver_port=36665
--webserver_interface=127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:58.327840 2354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:58.311605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:57013 (local address 127.8.203.126:42095)
0811 02:59:58.312281 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:58.312304 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:58.312745 (+ 441us) server_negotiation.cc:407] Connection header received
0811 02:59:58.314067 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:58.314078 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:58.314387 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:58.314684 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:58.315735 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.316879 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.318346 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.318806 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.318955 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:58.321877 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:58.321884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:58.321899 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:58.321932 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:58.323970 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.324627 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.324633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.324639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.324774 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.325166 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.325169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.325171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.325694 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:58.325817 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:58.326553 (+ 736us) server_negotiation.cc:299] Negotiation successful
0811 02:59:58.326850 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":117,"threads_started":1}
I20260811 02:59:58.329748 2355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:58.311735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42095 (local address 127.8.203.65:57013)
0811 02:59:58.312563 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:58.312615 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:58.313639 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:58.314778 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:58.314786 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:58.315097 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:58.315568 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:58.315581 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.317040 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.317044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:58.318224 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:58.318235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.318960 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.318964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:58.319147 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:58.319739 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:58.319770 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:58.321655 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:58.324117 (+ 2462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:58.324123 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:58.324137 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:58.324487 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:58.324887 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:58.324891 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:58.324893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:58.325058 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:58.325890 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:58.325896 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:58.326188 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 02:59:58.328554 (+ 2366us) client_negotiation.cc:240] Negotiation successful
0811 02:59:58.328858 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":133,"threads_started":1}
I20260811 02:59:58.332549 2348 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42095
I20260811 02:59:58.332774 2348 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:58.333271 2348 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
W20260811 02:59:58.335242 2348 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:42095 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:59:58.335368 2348 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:58.335546 2348 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:58.336313 2348 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:58.336479 2348 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
W20260811 02:59:58.337153 2348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:59:58.497020 2359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:58.497413 2359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:58.497488 2359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:58.503337 2359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:58.503424 2359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:58.503513 2359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 02:59:58.511538 2359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:37163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=36665
--tserver_master_addrs=127.8.203.126:42095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:58.512935 2359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:58.514238 2359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:58.526762 2368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:58.526767 2370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:58.526852 2374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:58.527082 2359 server_base.cc:1034] running on GCE node
I20260811 02:59:58.527607 2359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:58.528241 2359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:58.529356 2359 hybrid_clock.cc:648] HybridClock initialized: now 1786417198529322 us; error 48 us; skew 500 ppm
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:58.532130 2359 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 02:59:58.533043 2359 webserver.cc:460] Webserver started at http://127.8.203.66:36665/ using document root <none> and password file <none>
I20260811 02:59:58.533640 2359 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:58.533717 2359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:58.554801 2359 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:59:58.557473 2383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.558784 2359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:59:58.558890 2359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "54fe9a68171b4f8ca9c611147bd773e8"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "33e5f5755221a5e6baf9e09249d669e2"
server_key_iv: "4d1e4031b921913c38e43f63ca3ce9cf"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:58.559466 2359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:58.589258 2359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:58.592219 2359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:58.592404 2359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:58.592993 2359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:58.593894 2359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:58.593957 2359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.594020 2359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:58.594050 2359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.605907 2359 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:37163
I20260811 02:59:58.606863 2599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:37163 every 8 connection(s)
I20260811 02:59:58.607074 2359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:58.617164 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 2359
I20260811 02:59:58.617690 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.203.84:43473/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:34915
--local_ip_for_outbound_sockets=127.8.203.67
--tserver_master_addrs=127.8.203.126:42095
--webserver_port=44507
--webserver_interface=127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:59:58.621507 2354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:58.609129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:36051 (local address 127.8.203.126:42095)
0811 02:59:58.609316 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:58.609322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:58.610200 (+ 878us) server_negotiation.cc:407] Connection header received
0811 02:59:58.611180 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:58.611184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:58.611246 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:58.611319 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:58.612572 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.613188 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.614316 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.614645 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.614744 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:58.617410 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:58.617414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:58.617417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:58.617447 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:58.619117 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.619722 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.619726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.619728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.619816 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.620215 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.620219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.620221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.620471 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:58.620575 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:58.621171 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 02:59:58.621331 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:59:58.622367 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:58.609554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42095 (local address 127.8.203.66:36051)
0811 02:59:58.610043 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:58.610095 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:58.610972 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:58.611513 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:58.611520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:58.611831 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:58.612430 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:58.612447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.613330 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.613334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:58.614196 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:58.614207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.614798 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.614802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:58.614998 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:58.615596 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:58.615618 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:58.617169 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:58.619264 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:58.619270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:58.619285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:58.619601 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:58.619920 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:58.619922 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:58.619925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:58.620088 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:58.620638 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:58.620645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:58.620922 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 02:59:58.621087 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 02:59:58.621438 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260811 02:59:58.623561 2601 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42095
I20260811 02:59:58.623826 2601 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:58.624437 2601 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
W20260811 02:59:58.626145 2601 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:42095 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:59:58.626263 2601 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:58.626426 2601 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:58.627238 2601 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:58.627429 2601 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
W20260811 02:59:58.628222 2601 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:59:58.768071 2618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:59:58.768519 2618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:59:58.768646 2618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:59:58.773958 2618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:59:58.774017 2618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:59:58.774096 2618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 02:59:58.780897 2618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.203.84:43473/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:34915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=44507
--tserver_master_addrs=127.8.203.126:42095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:59:58.782320 2618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:59:58.783428 2618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:59:58.792912 2632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:59:58.793007 2631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:58.793087 2618 server_base.cc:1034] running on GCE node
W20260811 02:59:58.793360 2634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:59:58.793772 2618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:59:58.794414 2618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:59:58.795591 2618 hybrid_clock.cc:648] HybridClock initialized: now 1786417198795570 us; error 38 us; skew 500 ppm
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:58.798627 2618 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 02:59:58.799744 2618 webserver.cc:460] Webserver started at http://127.8.203.67:44507/ using document root <none> and password file <none>
I20260811 02:59:58.800496 2618 fs_manager.cc:359] Metadata directory not provided
I20260811 02:59:58.800595 2618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:58.822796 2618 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.000s
I20260811 02:59:58.826020 2670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.827646 2618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:59:58.827777 2618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "eeb1b373865041598f22148dbad7e85a"
format_stamp: "Formatted at 2026-08-11 02:59:57 on dist-test-slave-gjw7"
server_key: "cb22fc4d8b4a3224aa98a959be6616dd"
server_key_iv: "38a52ce3992009147c9c2bdd177a4fe4"
server_key_version: "kuduclusterkey@0"
I20260811 02:59:58.828508 2618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:59:58.849851 2618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:59:58.853039 2618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:59:58.853255 2618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:59:58.853981 2618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:59:58.855017 2618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:59:58.855083 2618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.855137 2618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:59:58.855170 2618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:59:58.866992 2618 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:34915
I20260811 02:59:58.867266 2897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:34915 every 8 connection(s)
I20260811 02:59:58.868000 2618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 02:59:58.870257 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 2618
Aug 11 02:59:58 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 02:59:58.884497 2354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:58.871925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56423 (local address 127.8.203.126:42095)
0811 02:59:58.872062 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:58.872067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:58.872083 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:59:58.872487 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:58.872491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:58.872546 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:58.872625 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:58.874103 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.874803 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.876487 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.876816 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.876897 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:58.880184 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:58.880188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:58.880191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:58.880219 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:58.881974 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.882609 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.882613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.882614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.882700 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.883138 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.883142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.883144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.883383 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:58.883459 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:58.884187 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 02:59:58.884312 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:59:58.886412 2905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:58.870773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42095 (local address 127.8.203.67:56423)
0811 02:59:58.871324 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:58.871368 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:58.872292 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:58.872865 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:58.872875 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:58.873226 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:59:58.873945 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:58.873967 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.874957 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.874960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:58.876329 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:58.876341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.877107 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.877112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:58.877339 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:58.878046 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:59:58.878067 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:59:58.879987 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:59:58.882113 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:58.882119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:58.882136 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:58.882490 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:58.882809 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:59:58.882812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:59:58.882814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:59:58.883014 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:59:58.883548 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:59:58.883555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:59:58.883900 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 02:59:58.884958 (+ 1058us) client_negotiation.cc:240] Negotiation successful
0811 02:59:58.885328 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":135,"threads_started":1}
I20260811 02:59:58.887990 2899 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42095
I20260811 02:59:58.888236 2899 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:58.888815 2899 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:58.889880 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:58.871927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.8.203.126:42095)
0811 02:59:58.872665 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:58.872672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:58.872684 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:59:58.872714 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:58.872717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:58.872750 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:58.872820 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:58.873602 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.874192 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.883081 (+ 8889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:58.883443 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:58.883552 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:58.885324 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:58.885328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:58.885331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:58.885356 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:58.887418 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.887929 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.887933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.887935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.888023 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:58.888356 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:58.888360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:58.888361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:58.888580 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:58.888804 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:58.889578 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 02:59:58.889719 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":676,"thread_start_us":100,"threads_started":1}
I20260811 02:59:58.890352 1955 ts_manager.cc:194] Registered new tserver with Master: eeb1b373865041598f22148dbad7e85a (127.8.203.67:34915)
I20260811 02:59:58.891891 1955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:56423
Aug 11 02:59:59 dist-test-slave-gjw7 krb5kdc[32364](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786417199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:59:59.337400 2348 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:59.337697 2348 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:59.338347 1955 ts_manager.cc:194] Registered new tserver with Master: 2861acdd94f448ad8514a9bab7e41433 (127.8.203.65:44761)
I20260811 02:59:59.338932 1955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:57013
Aug 11 02:59:59 dist-test-slave-gjw7 krb5kdc[32364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for HTTP/127.8.203.84@KRBTEST.COM
I20260811 02:59:59.628448 2601 heartbeater.cc:464] Registering TS with master...
I20260811 02:59:59.628734 2601 heartbeater.cc:511] Master 127.8.203.126:42095 requested a full tablet report, sending...
I20260811 02:59:59.629422 1955 ts_manager.cc:194] Registered new tserver with Master: 54fe9a68171b4f8ca9c611147bd773e8 (127.8.203.66:37163)
I20260811 02:59:59.630057 1955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:36051
I20260811 02:59:59.638705 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:59:59.651464 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.640468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42142 (local address 127.8.203.126:42095)
0811 02:59:59.641155 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:59.641159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:59.641616 (+ 457us) server_negotiation.cc:407] Connection header received
0811 02:59:59.641661 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:59.641664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:59.641716 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:59.641800 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:59:59.642946 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.643764 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.644954 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.645331 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.645427 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.647270 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:59:59.647274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:59:59.647277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:59:59.647305 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:59:59.649171 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:59.649682 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:59.649686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:59.649688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:59.649787 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:59:59.650129 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:59:59.650132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:59:59.650134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:59:59.650357 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:59:59.650480 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:59.651149 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 02:59:59.651275 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":121,"threads_started":1}
I20260811 02:59:59.655936 1955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:59:59.660974 1955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:59:59.689834 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.682410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.8.203.67:34915)
0811 02:59:59.682853 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:59.682861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:59.683738 (+ 877us) server_negotiation.cc:407] Connection header received
0811 02:59:59.684050 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:59.684057 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:59.684278 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:59.684415 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:59.684966 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.686086 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.687991 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.688844 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.688925 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.689081 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:59.689566 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 02:59:59.689676 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":89,"threads_started":1}
I20260811 02:59:59.690352 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.681995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:34915 (local address 127.0.0.1:57442)
0811 02:59:59.683581 (+ 1586us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:59.683635 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:59.683856 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:59.684635 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:59.684642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:59.684679 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:59.684828 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.684835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.686226 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.686229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.687571 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.687583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.689187 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.689191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.689326 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.689370 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 02:59:59.690060 (+ 690us) client_negotiation.cc:240] Negotiation successful
0811 02:59:59.690199 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1447,"spinlock_wait_cycles":4480,"thread_start_us":87,"threads_started":1}
I20260811 02:59:59.691730 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.681941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:44761 (local address 127.0.0.1:49930)
0811 02:59:59.683744 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:59.683759 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:59.683858 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:59.684635 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:59.684642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:59.684691 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:59.684830 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.684837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.687252 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.687255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.688478 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.688490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.691350 (+ 2860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.691354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.691451 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.691477 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:59:59.691530 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 02:59:59.691597 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"spinlock_wait_cycles":26368,"thread_start_us":94,"threads_started":1}
I20260811 02:59:59.692620 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.683254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.8.203.65:44761)
0811 02:59:59.684137 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:59.684144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:59.684168 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:59:59.684245 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:59.684250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:59.684462 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:59.684989 (+ 527us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:59.685149 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.687117 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.688639 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.691202 (+ 2563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.692173 (+ 971us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.692322 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:59.692385 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 02:59:59.692481 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":102,"threads_started":1}
I20260811 02:59:59.694000 2753 tablet_service.cc:1467] Processing CreateTablet for tablet 62e3d7f1090b49cea93ea83f9805a0a2 (DEFAULT_TABLE table=test-table [id=e180327d709543459446329a84f49f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:59:59.694689 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.682789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:37163 (local address 127.0.0.1:58944)
0811 02:59:59.687796 (+ 5007us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:59.687808 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:59.687885 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:59.690503 (+ 2618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:59.690506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:59.690522 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:59.690602 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.690608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.692004 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.692007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.693062 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.693074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.694300 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.694303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.694385 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.694420 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:59:59.694498 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 02:59:59.694566 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4941,"thread_start_us":104,"threads_started":1}
I20260811 02:59:59.694787 2944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.683124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.8.203.66:37163)
0811 02:59:59.689873 (+ 6749us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:59.689880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:59.689905 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:59:59.689989 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:59.689995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:59.690201 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:59.690353 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:59.690743 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.691868 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.693218 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.693999 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.694103 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.694237 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:59.694553 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 02:59:59.694653 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6633,"thread_start_us":100,"threads_started":1}
I20260811 02:59:59.695514 2753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e3d7f1090b49cea93ea83f9805a0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:59.696233 2212 tablet_service.cc:1467] Processing CreateTablet for tablet 62e3d7f1090b49cea93ea83f9805a0a2 (DEFAULT_TABLE table=test-table [id=e180327d709543459446329a84f49f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:59:59.697234 2212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e3d7f1090b49cea93ea83f9805a0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:59.697237 2477 tablet_service.cc:1467] Processing CreateTablet for tablet 62e3d7f1090b49cea93ea83f9805a0a2 (DEFAULT_TABLE table=test-table [id=e180327d709543459446329a84f49f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:59:59.698256 2477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62e3d7f1090b49cea93ea83f9805a0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:59:59.706624 2952 tablet_bootstrap.cc:492] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Bootstrap starting.
I20260811 02:59:59.706964 2957 tablet_bootstrap.cc:492] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Bootstrap starting.
I20260811 02:59:59.709188 2957 tablet_bootstrap.cc:654] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:59.709539 2952 tablet_bootstrap.cc:654] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:59.710700 2953 tablet_bootstrap.cc:492] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433: Bootstrap starting.
I20260811 02:59:59.711082 2952 log.cc:824] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:59.712488 2957 log.cc:824] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:59.713645 2952 tablet_bootstrap.cc:492] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: No bootstrap required, opened a new log
I20260811 02:59:59.713856 2952 ts_tablet_manager.cc:1397] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:59:59.714859 2953 tablet_bootstrap.cc:654] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433: Neither blocks nor log segments found. Creating new log.
I20260811 02:59:59.715996 2953 log.cc:824] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433: Log is configured to *not* fsync() on all Append() calls
I20260811 02:59:59.719086 2957 tablet_bootstrap.cc:492] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: No bootstrap required, opened a new log
I20260811 02:59:59.719280 2957 ts_tablet_manager.cc:1397] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.001s
I20260811 02:59:59.720482 2952 raft_consensus.cc:348] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.720881 2952 raft_consensus.cc:374] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:59.721004 2952 raft_consensus.cc:729] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeb1b373865041598f22148dbad7e85a, State: Initialized, Role: FOLLOWER
I20260811 02:59:59.721630 2952 consensus_queue.cc:260] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.723297 2952 ts_tablet_manager.cc:1428] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Time spent starting tablet: real 0.009s user 0.003s sys 0.005s
I20260811 02:59:59.723728 2899 heartbeater.cc:503] Master 127.8.203.126:42095 was elected leader, sending a full tablet report...
I20260811 02:59:59.725075 2957 raft_consensus.cc:348] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.725441 2957 raft_consensus.cc:374] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:59.725533 2957 raft_consensus.cc:729] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fe9a68171b4f8ca9c611147bd773e8, State: Initialized, Role: FOLLOWER
I20260811 02:59:59.726069 2957 consensus_queue.cc:260] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.727274 2957 ts_tablet_manager.cc:1428] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260811 02:59:59.729363 2953 tablet_bootstrap.cc:492] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433: No bootstrap required, opened a new log
I20260811 02:59:59.729866 2601 heartbeater.cc:503] Master 127.8.203.126:42095 was elected leader, sending a full tablet report...
I20260811 02:59:59.732275 2953 ts_tablet_manager.cc:1397] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.000s
I20260811 02:59:59.739573 2953 raft_consensus.cc:348] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.739948 2953 raft_consensus.cc:374] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:59:59.740041 2953 raft_consensus.cc:729] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2861acdd94f448ad8514a9bab7e41433, State: Initialized, Role: FOLLOWER
I20260811 02:59:59.740600 2953 consensus_queue.cc:260] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.741979 2953 ts_tablet_manager.cc:1428] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433: Time spent starting tablet: real 0.010s user 0.000s sys 0.008s
I20260811 02:59:59.742380 2348 heartbeater.cc:503] Master 127.8.203.126:42095 was elected leader, sending a full tablet report...
I20260811 02:59:59.765941 2965 raft_consensus.cc:482] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:59:59.766206 2965 raft_consensus.cc:504] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.767735 2965 leader_election.cc:288] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2861acdd94f448ad8514a9bab7e41433 (127.8.203.65:44761), 54fe9a68171b4f8ca9c611147bd773e8 (127.8.203.66:37163)
I20260811 02:59:59.775511 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.769723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:37263 (local address 127.8.203.65:44761)
0811 02:59:59.769838 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:59.769842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:59.770076 (+ 234us) server_negotiation.cc:407] Connection header received
0811 02:59:59.770301 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:59.770305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:59.770359 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:59.770409 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:59.771103 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.771888 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.774219 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.774779 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.774881 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.774935 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:59.775303 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 02:59:59.775381 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 02:59:59.776320 2972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.768300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:44761 (local address 127.8.203.67:37263)
0811 02:59:59.769973 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:59.769987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:59.770163 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:59.770571 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:59.770574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:59.770594 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:59.770952 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.770965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.772042 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.772046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.773671 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.773683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.775030 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.775034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.775164 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.775188 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:59:59.775981 (+ 793us) client_negotiation.cc:240] Negotiation successful
0811 02:59:59.776055 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1601,"thread_start_us":225,"threads_started":1}
I20260811 02:59:59.777393 2231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e3d7f1090b49cea93ea83f9805a0a2" candidate_uuid: "eeb1b373865041598f22148dbad7e85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2861acdd94f448ad8514a9bab7e41433" is_pre_election: true
I20260811 02:59:59.777769 2231 raft_consensus.cc:2454] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eeb1b373865041598f22148dbad7e85a in term 0.
I20260811 02:59:59.779547 2681 leader_election.cc:302] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2861acdd94f448ad8514a9bab7e41433, eeb1b373865041598f22148dbad7e85a; no voters:
I20260811 02:59:59.779896 2965 raft_consensus.cc:2781] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:59:59.779980 2965 raft_consensus.cc:482] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:59:59.780025 2965 raft_consensus.cc:3037] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:59.781327 2944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.769505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:60547 (local address 127.8.203.66:37163)
0811 02:59:59.769625 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:59.769630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:59.774068 (+ 4438us) server_negotiation.cc:407] Connection header received
0811 02:59:59.774189 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:59.774192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:59.774241 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:59.774305 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:59:59.774806 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.775481 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.778702 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.779231 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.779315 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.779380 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:59.781091 (+ 1711us) server_negotiation.cc:299] Negotiation successful
0811 02:59:59.781174 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 02:59:59.781855 2971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.768813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:37163 (local address 127.8.203.67:60547)
0811 02:59:59.773977 (+ 5164us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:59.773992 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:59.774101 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:59.774421 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:59:59.774424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:59:59.774440 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:59:59.774668 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.774675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.775614 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.775617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.778569 (+ 2952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:59:59.778579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.779469 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.779472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:59:59.779607 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.779631 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:59:59.781535 (+ 1904us) client_negotiation.cc:240] Negotiation successful
0811 02:59:59.781606 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5094,"mutex_wait_us":1,"thread_start_us":165,"threads_started":1}
I20260811 02:59:59.781934 2965 raft_consensus.cc:504] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.782312 2965 leader_election.cc:288] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [CANDIDATE]: Term 1 election: Requested vote from peers 2861acdd94f448ad8514a9bab7e41433 (127.8.203.65:44761), 54fe9a68171b4f8ca9c611147bd773e8 (127.8.203.66:37163)
I20260811 02:59:59.782753 2495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e3d7f1090b49cea93ea83f9805a0a2" candidate_uuid: "eeb1b373865041598f22148dbad7e85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fe9a68171b4f8ca9c611147bd773e8" is_pre_election: true
I20260811 02:59:59.783070 2495 raft_consensus.cc:2454] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eeb1b373865041598f22148dbad7e85a in term 0.
I20260811 02:59:59.784523 2495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e3d7f1090b49cea93ea83f9805a0a2" candidate_uuid: "eeb1b373865041598f22148dbad7e85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fe9a68171b4f8ca9c611147bd773e8"
I20260811 02:59:59.784639 2495 raft_consensus.cc:3037] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:59.786476 2495 raft_consensus.cc:2454] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eeb1b373865041598f22148dbad7e85a in term 1.
I20260811 02:59:59.786871 2681 leader_election.cc:302] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fe9a68171b4f8ca9c611147bd773e8, eeb1b373865041598f22148dbad7e85a; no voters:
I20260811 02:59:59.787170 2231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62e3d7f1090b49cea93ea83f9805a0a2" candidate_uuid: "eeb1b373865041598f22148dbad7e85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2861acdd94f448ad8514a9bab7e41433"
I20260811 02:59:59.787307 2231 raft_consensus.cc:3037] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:59:59.789129 2231 raft_consensus.cc:2454] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eeb1b373865041598f22148dbad7e85a in term 1.
I20260811 02:59:59.789691 2965 raft_consensus.cc:2781] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:59:59.789954 2965 raft_consensus.cc:686] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 1 LEADER]: Becoming Leader. State: Replica: eeb1b373865041598f22148dbad7e85a, State: Running, Role: LEADER
I20260811 02:59:59.790331 2965 consensus_queue.cc:237] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } }
I20260811 02:59:59.794291 1955 catalog_manager.cc:5495] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a reported cstate change: term changed from 0 to 1, leader changed from <none> to eeb1b373865041598f22148dbad7e85a (127.8.203.67). New cstate: current_term: 1 leader_uuid: "eeb1b373865041598f22148dbad7e85a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eeb1b373865041598f22148dbad7e85a" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 } health_report { overall_health: UNKNOWN } } }
I20260811 02:59:59.819011 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.813958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57452 (local address 127.8.203.67:34915)
0811 02:59:59.814108 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:59:59.814112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:59:59.814128 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:59:59.814211 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:59:59.814214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:59:59.814281 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:59:59.814406 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:59:59.814716 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.815501 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.816743 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:59:59.817094 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:59:59.817201 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:59:59.817667 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:59:59.818247 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:59:59.818371 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 02:59:59.818792 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 02:59:59.818868 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 02:59:59.820742 2349 tablet.cc:2369] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:59:59.827251 2495 raft_consensus.cc:1261] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 1 FOLLOWER]: Refusing update from remote peer eeb1b373865041598f22148dbad7e85a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:59:59.828269 2974 consensus_queue.cc:1035] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fe9a68171b4f8ca9c611147bd773e8" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:59:59.835879 2231 raft_consensus.cc:1261] T 62e3d7f1090b49cea93ea83f9805a0a2 P 2861acdd94f448ad8514a9bab7e41433 [term 1 FOLLOWER]: Refusing update from remote peer eeb1b373865041598f22148dbad7e85a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:59:59.836632 2974 consensus_queue.cc:1035] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2861acdd94f448ad8514a9bab7e41433" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 44761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:59:59.853410 2981 mvcc.cc:205] Tried to move back new op lower bound from 7317164850482245632 to 7317164850346057728. Current Snapshot: MvccSnapshot[applied={T|T < 7317164850482245632}]
I20260811 02:59:59.855712 2978 mvcc.cc:205] Tried to move back new op lower bound from 7317164850482245632 to 7317164850346057728. Current Snapshot: MvccSnapshot[applied={T|T < 7317164850482245632 or (T in {7317164850482245632})}]
I20260811 02:59:59.859486 2979 mvcc.cc:205] Tried to move back new op lower bound from 7317164850482245632 to 7317164850346057728. Current Snapshot: MvccSnapshot[applied={T|T < 7317164850482245632}]
I20260811 02:59:59.863991 1952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:59:59.864192 1952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:59:59.867425 1952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2af679b973704ff4962e294e468e4567: Sending DeleteTablet for 3 replicas of tablet 62e3d7f1090b49cea93ea83f9805a0a2
I20260811 02:59:59.868423 2753 tablet_service.cc:1514] Processing DeleteTablet for tablet 62e3d7f1090b49cea93ea83f9805a0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:59:59 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:57442
I20260811 02:59:59.869570 2212 tablet_service.cc:1514] Processing DeleteTablet for tablet 62e3d7f1090b49cea93ea83f9805a0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:59:59 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:49930
I20260811 02:59:59.870939 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 2026
I20260811 02:59:59.871832 2477 tablet_service.cc:1514] Processing DeleteTablet for tablet 62e3d7f1090b49cea93ea83f9805a0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:59:59 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:58944
I20260811 02:59:59.873538 2988 tablet_replica.cc:330] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: stopping tablet replica
I20260811 02:59:59.873863 2988 raft_consensus.cc:2229] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:59:59.874228 2988 raft_consensus.cc:2258] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:59:59.882640 2990 tablet_replica.cc:330] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: stopping tablet replica
I20260811 02:59:59.883071 2990 raft_consensus.cc:2229] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:59:59.888054 2990 raft_consensus.cc:2258] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:59:59.889978 2990 ts_tablet_manager.cc:1905] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:59:59.891785 2988 ts_tablet_manager.cc:1905] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:59:59.897456 2988 ts_tablet_manager.cc:1918] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:59:59.897672 2988 log.cc:1196] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/62e3d7f1090b49cea93ea83f9805a0a2
I20260811 02:59:59.898087 2988 ts_tablet_manager.cc:1939] T 62e3d7f1090b49cea93ea83f9805a0a2 P eeb1b373865041598f22148dbad7e85a: Deleting consensus metadata
I20260811 02:59:59.899441 1933 catalog_manager.cc:4841] TS eeb1b373865041598f22148dbad7e85a (127.8.203.67:34915): tablet 62e3d7f1090b49cea93ea83f9805a0a2 (table test-table [id=e180327d709543459446329a84f49f3e]) successfully deleted
I20260811 02:59:59.900499 2990 ts_tablet_manager.cc:1918] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 02:59:59.904937 1931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:44761 (error 108)
I20260811 02:59:59.905393 2990 log.cc:1196] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/62e3d7f1090b49cea93ea83f9805a0a2
I20260811 02:59:59.905977 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 2359
I20260811 02:59:59.906035 2990 ts_tablet_manager.cc:1939] T 62e3d7f1090b49cea93ea83f9805a0a2 P 54fe9a68171b4f8ca9c611147bd773e8: Deleting consensus metadata
I20260811 02:59:59.914925 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:59:59.914545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:44761 (local address 127.0.0.1:49936)
0811 02:59:59.914752 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:59.914826 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:44761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260811 02:59:59.915354 1931 catalog_manager.cc:4584] TS 2861acdd94f448ad8514a9bab7e41433 (127.8.203.65:44761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62e3d7f1090b49cea93ea83f9805a0a2: Network error: Client connection negotiation failed: client connection to 127.8.203.65:44761: connect: Connection refused (error 111)
W20260811 02:59:59.921484 2939 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:59:59.920783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:37163 (local address 127.0.0.1:58960)
0811 02:59:59.920944 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 02:59:59.920958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:59:59.921047 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:59:59.921376 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.66:37163: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 02:59:59.922205 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 2618
I20260811 02:59:59.940440 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 1893
I20260811 02:59:59.955736 9005 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 03:00:00.304291 9005 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 03:00:00.304417 9005 mini_ranger.cc:66] Stopping Ranger...
I20260811 03:00:00.651265 9005 mini_ranger.cc:68] Stopped Ranger
2026-08-11 03:00:00.651 UTC [32427] LOG: received smart shutdown request
2026-08-11 03:00:00.655 UTC [32427] LOG: background worker "logical replication launcher" (PID 32442) exited with exit code 1
2026-08-11 03:00:00.655 UTC [32435] LOG: shutting down
2026-08-11 03:00:00.782 UTC [32427] LOG: database system is shut down
2026-08-11T03:00:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46708 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:00:00 dist-test-slave-gjw7 krb5kdc[3008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:00:00 dist-test-slave-gjw7 krb5kdc[3008](info): set up 2 sockets
Aug 11 03:00:00 dist-test-slave-gjw7 krb5kdc[3008](info): commencing operation
krb5kdc: starting...
W20260811 03:00:02.093765 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:00:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:00:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:00:02.211925 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36479
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:36479,127.8.203.125:33985,127.8.203.124:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:02.322808 3029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:02.323058 3029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:02.323115 3029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:02.327018 3029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:02.327069 3029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:02.327095 3029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:02.327117 3029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:02.327145 3029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:02.332629 3029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:36479,127.8.203.125:33985,127.8.203.124:40249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:02.333668 3029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:02.334645 3029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:02.342561 3040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:02.342567 3045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:02.342567 3043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:02.342758 3029 server_base.cc:1034] running on GCE node
I20260811 03:00:02.343372 3029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:02.344415 3029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:02.345566 3029 hybrid_clock.cc:648] HybridClock initialized: now 1786417202345551 us; error 26 us; skew 500 ppm
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:02.348142 3029 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:00:02.349010 3029 webserver.cc:460] Webserver started at http://127.8.203.126:36227/ using document root <none> and password file <none>
I20260811 03:00:02.349581 3029 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:02.349649 3029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:02.349854 3029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:02.351891 3029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "ade5238a7fdb60d57ead268867b03eb6"
server_key_iv: "77a7ae0f6e488e35f0a1de134fbed669"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.352432 3029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "ade5238a7fdb60d57ead268867b03eb6"
server_key_iv: "77a7ae0f6e488e35f0a1de134fbed669"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.355970 3029 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 03:00:02.358289 3060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:02.359428 3029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:00:02.359550 3029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "ade5238a7fdb60d57ead268867b03eb6"
server_key_iv: "77a7ae0f6e488e35f0a1de134fbed669"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.359642 3029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:02.377939 3029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:02.380975 3029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:02.381150 3029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:02.388073 3029 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:36479
I20260811 03:00:02.388089 3157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:36479 every 8 connection(s)
I20260811 03:00:02.389016 3029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:00:02.391731 3160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:02.395387 3160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:00:02.397433 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 3029
I20260811 03:00:02.397558 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:00:02.397819 9005 external_mini_cluster.cc:1428] Setting key 87cf09a055f14aff54870ca24d9a149c
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.203.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.125@KRBTEST.COM" created.
I20260811 03:00:02.410472 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.397666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.8.203.126:36479)
0811 03:00:02.398112 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.398130 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.398453 (+ 323us) server_negotiation.cc:407] Connection header received
0811 03:00:02.399439 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.399445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.399682 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.399862 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.400834 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.401748 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.403129 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.403592 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.403753 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.406208 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.406214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.406224 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.406245 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.407771 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.408247 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.408251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.408253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.408355 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.408603 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.408605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.408607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.408958 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.409077 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.409449 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.409737 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":291,"thread_start_us":144,"threads_started":1}
I20260811 03:00:02.410477 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.397666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:36479 (local address 127.0.0.1:41310)
0811 03:00:02.398292 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.398342 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.399132 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.399902 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.399908 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.400356 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.400694 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.400713 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.401878 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.401882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.403015 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.403028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.403737 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.403740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.403835 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.404351 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.404367 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.406066 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.407878 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.407887 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.407895 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.408158 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.408433 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.408435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.408437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.408540 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.409052 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.409057 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.409180 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.409361 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.409642 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"spinlock_wait_cycles":53120,"thread_start_us":151,"threads_started":1}
I20260811 03:00:02.413203 3160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:00:02.413923 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.413689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36012)
0811 03:00:02.413798 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.413870 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 03:00:02.414116 3064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
I20260811 03:00:02.414932 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.414736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36014)
0811 03:00:02.414839 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.414882 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 03:00:02.415061 3160 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:33985: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab.
Entry for principal kudu/127.8.203.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab.
I20260811 03:00:02.423795 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:33985
--webserver_interface=127.8.203.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab
--principal=kudu/127.8.203.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:36479,127.8.203.125:33985,127.8.203.124:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:02.464303 3160 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } attempt: 1
I20260811 03:00:02.465164 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.464871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36016)
0811 03:00:02.465022 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.465085 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:00:02.465962 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.465743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36028)
0811 03:00:02.465854 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.465901 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 03:00:02.466100 3160 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:33985: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
I20260811 03:00:02.534242 3160 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } attempt: 2
I20260811 03:00:02.535068 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.534778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36042)
0811 03:00:02.534926 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.534989 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 03:00:02.535665 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.535493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36046)
0811 03:00:02.535569 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.535610 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 03:00:02.535785 3160 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:33985: Network error: Client connection negotiation failed: client connection to 127.8.203.125:33985: connect: Connection refused (error 111)
W20260811 03:00:02.535827 3176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:02.536110 3176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:02.536175 3176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:02.540146 3176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:02.540197 3176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:02.540215 3176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:02.540234 3176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:02.540262 3176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:02.545404 3176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:36479,127.8.203.125:33985,127.8.203.124:40249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab
--principal=kudu/127.8.203.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:33985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.203.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:02.546401 3176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:02.547379 3176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:02.555670 3184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:02.555675 3183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:02.555728 3176 server_base.cc:1034] running on GCE node
W20260811 03:00:02.555686 3186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:02.556468 3176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:02.557592 3176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:02.558746 3176 hybrid_clock.cc:648] HybridClock initialized: now 1786417202558730 us; error 27 us; skew 500 ppm
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:02.561442 3176 init.cc:377] Logged in from keytab as kudu/127.8.203.125@KRBTEST.COM (short username kudu)
I20260811 03:00:02.562279 3176 webserver.cc:460] Webserver started at http://127.8.203.125:34693/ using document root <none> and password file <none>
I20260811 03:00:02.562784 3176 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:02.562839 3176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:02.563051 3176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:02.565073 3176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data/instance:
uuid: "e67af106d5d841df8aa505522513485a"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "c9ce1178f890d2fd673d9f1ad01eda3d"
server_key_iv: "a6e973518710deedea98d92da05368d7"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.565610 3176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/wal/instance:
uuid: "e67af106d5d841df8aa505522513485a"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "c9ce1178f890d2fd673d9f1ad01eda3d"
server_key_iv: "a6e973518710deedea98d92da05368d7"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.569128 3176 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:00:02.571513 3203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:02.572685 3176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:02.572791 3176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/wal
uuid: "e67af106d5d841df8aa505522513485a"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "c9ce1178f890d2fd673d9f1ad01eda3d"
server_key_iv: "a6e973518710deedea98d92da05368d7"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.572902 3176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:02.584110 3176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:02.586753 3176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:02.586930 3176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:02.593547 3176 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.125:33985
I20260811 03:00:02.593577 3316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.125:33985 every 8 connection(s)
I20260811 03:00:02.594480 3176 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
I20260811 03:00:02.597175 3318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:02.599684 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 3176
I20260811 03:00:02.599789 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-1/wal/instance
I20260811 03:00:02.600039 9005 external_mini_cluster.cc:1428] Setting key e3e43b52d2baf8d74d17b530fa34f017
I20260811 03:00:02.601030 3318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.203.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.124@KRBTEST.COM" created.
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:02.615880 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.603218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.8.203.126:36479)
0811 03:00:02.603380 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.603386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.604282 (+ 896us) server_negotiation.cc:407] Connection header received
0811 03:00:02.605198 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.605202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.605261 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.605370 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.606543 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.607079 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.608254 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.608502 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.608596 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.612209 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.612212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.612214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.612242 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.613817 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.614320 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.614323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.614325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.614421 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.614775 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.614777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.614778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.614955 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.615028 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.615644 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.615752 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 03:00:02.617095 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.603627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:36479 (local address 127.0.0.1:41324)
0811 03:00:02.604135 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.604178 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.604993 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.605536 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.605543 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.605897 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.606394 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.606409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.607203 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.607206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.608131 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.608142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.608651 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.608655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.608844 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.609399 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.609416 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.612038 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.613932 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.613937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.613951 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.614220 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.614515 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.614518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.614519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.614696 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.615052 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.615059 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.615392 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.615594 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.616006 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":155,"threads_started":1}
I20260811 03:00:02.618628 3318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
I20260811 03:00:02.626950 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.619156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36052)
0811 03:00:02.619265 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.619277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.619338 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.619907 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.619909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.620063 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.620146 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.620151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.621064 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.621067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.621686 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.621696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.622296 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.622299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.622391 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.622789 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.622800 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.623937 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.625605 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.625608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.625609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.625792 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.626042 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.626044 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.626045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.626114 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.626506 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.626507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.626587 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.626692 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.626796 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 03:00:02.626930 3160 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } attempt: 3
I20260811 03:00:02.627030 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.619263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36052 (local address 127.8.203.125:33985)
0811 03:00:02.619588 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.619596 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.619612 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:00:02.619641 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.619645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.619788 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.619902 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.620242 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.620962 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.621809 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.622190 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.622278 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.624072 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.624081 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.624092 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.624114 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.625518 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.625892 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.625897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.625898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.625974 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.626173 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.626175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.626176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.626428 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.626527 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.626765 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.626884 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":160,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab.
Entry for principal kudu/127.8.203.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab.
I20260811 03:00:02.628424 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:40249
--webserver_interface=127.8.203.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab
--principal=kudu/127.8.203.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:36479,127.8.203.125:33985,127.8.203.124:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:02.629156 3318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:00:02.629812 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.629588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56164)
0811 03:00:02.629690 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.629760 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 03:00:02.630062 3209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
I20260811 03:00:02.631759 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.631285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56176)
0811 03:00:02.631651 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.631701 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297}
W20260811 03:00:02.631933 3318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:40249: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
I20260811 03:00:02.635610 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.627937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.8.203.125:33985)
0811 03:00:02.628057 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.628062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.628074 (+ 12us) server_negotiation.cc:407] Connection header received
0811 03:00:02.628106 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.628109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.628149 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.628227 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.629134 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.629799 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.630698 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.630941 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.631366 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.632942 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.632946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.632949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.632971 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.634187 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.634560 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.634563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.634565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.634634 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.634872 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.634875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.634877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.635034 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.635114 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.635382 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.635485 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 03:00:02.635612 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.627378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36060)
0811 03:00:02.627507 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.627522 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.627598 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.628271 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.628274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.628454 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.629023 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.629034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.629951 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.629955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.630579 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.630589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.631065 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.631069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.631172 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.631602 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.631613 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.632822 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.634284 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.634287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.634289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.634475 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.634714 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.634717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.634719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.634795 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.635121 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.635124 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.635227 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.635313 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.635453 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 03:00:02.636330 3160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:00:02.636966 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.636753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56188)
0811 03:00:02.636858 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.636908 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 03:00:02.637614 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.637390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56194)
0811 03:00:02.637519 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.637563 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 03:00:02.637739 3160 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:40249: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
I20260811 03:00:02.668887 3160 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } attempt: 1
I20260811 03:00:02.669699 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.669420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56200)
0811 03:00:02.669554 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.669617 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 03:00:02.670406 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.670191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56206)
0811 03:00:02.670290 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.670336 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 03:00:02.670538 3160 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:40249: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
I20260811 03:00:02.681185 3318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } attempt: 1
I20260811 03:00:02.681962 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.681654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56218)
0811 03:00:02.681812 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.681882 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 03:00:02.682646 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.682428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56226)
0811 03:00:02.682521 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.682579 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 03:00:02.682834 3318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:40249: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
W20260811 03:00:02.741572 3327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:02.741804 3327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:02.741842 3327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:02.745682 3327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:02.745728 3327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:02.745745 3327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:02.745764 3327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260811 03:00:02.745676 3160 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } attempt: 2
W20260811 03:00:02.745793 3327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:02.746429 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.746145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56242)
0811 03:00:02.746287 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.746345 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 03:00:02.747129 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.746952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56244)
0811 03:00:02.747038 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.747079 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 03:00:02.747242 3160 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:40249: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
I20260811 03:00:02.750989 3318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } attempt: 2
I20260811 03:00:02.751051 3327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:36479,127.8.203.125:33985,127.8.203.124:40249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab
--principal=kudu/127.8.203.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.203.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:02.751629 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.751382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56256)
0811 03:00:02.751514 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.751565 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 03:00:02.752229 3327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:02.752396 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.752156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56270)
0811 03:00:02.752261 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.752312 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 03:00:02.752538 3318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:40249: Network error: Client connection negotiation failed: client connection to 127.8.203.124:40249: connect: Connection refused (error 111)
I20260811 03:00:02.753278 3327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:02.761425 3348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:02.761452 3345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:02.761592 3346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:02.761603 3327 server_base.cc:1034] running on GCE node
I20260811 03:00:02.762262 3327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:02.763350 3327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:02.764504 3327 hybrid_clock.cc:648] HybridClock initialized: now 1786417202764476 us; error 37 us; skew 500 ppm
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:02.767083 3327 init.cc:377] Logged in from keytab as kudu/127.8.203.124@KRBTEST.COM (short username kudu)
I20260811 03:00:02.767943 3327 webserver.cc:460] Webserver started at http://127.8.203.124:43821/ using document root <none> and password file <none>
I20260811 03:00:02.768477 3327 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:02.768543 3327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:02.768755 3327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:02.770758 3327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data/instance:
uuid: "14be8ff9ce1d40b3ae10df81c234e821"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "d6f01e3c31a0b8ebeef307cb30b956cc"
server_key_iv: "2cb2842dc6bc0bbc40ebbde971c7f944"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.771284 3327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/wal/instance:
uuid: "14be8ff9ce1d40b3ae10df81c234e821"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "d6f01e3c31a0b8ebeef307cb30b956cc"
server_key_iv: "2cb2842dc6bc0bbc40ebbde971c7f944"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.774963 3327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:00:02.777220 3360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:02.778371 3327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:02.778491 3327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/wal
uuid: "14be8ff9ce1d40b3ae10df81c234e821"
format_stamp: "Formatted at 2026-08-11 03:00:02 on dist-test-slave-gjw7"
server_key: "d6f01e3c31a0b8ebeef307cb30b956cc"
server_key_iv: "2cb2842dc6bc0bbc40ebbde971c7f944"
server_key_version: "encryptionkey@0"
I20260811 03:00:02.778579 3327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:02.812062 3327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:02.814788 3327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:02.814963 3327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:02.821650 3327 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.124:40249
I20260811 03:00:02.821696 3472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.124:40249 every 8 connection(s)
I20260811 03:00:02.822561 3327 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
I20260811 03:00:02.825222 3473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:02.825860 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 3327
I20260811 03:00:02.825985 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786417039021389-9005-0/minicluster-data/master-2/wal/instance
I20260811 03:00:02.826282 9005 external_mini_cluster.cc:1428] Setting key fcda34161b8a92c1c4d92de11a937ce6
I20260811 03:00:02.829618 3473 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:02.840170 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.831273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.8.203.126:36479)
0811 03:00:02.831397 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.831401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.831706 (+ 305us) server_negotiation.cc:407] Connection header received
0811 03:00:02.831835 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.831839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.831883 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.831947 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.832639 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.833246 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.834192 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.834533 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.834614 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.836957 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.836960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.836962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.836984 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.838495 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.838997 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.839001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.839002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.839091 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.839388 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.839391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.839392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.839579 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.839653 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.839978 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.840062 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:00:02.840230 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.827773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56280 (local address 127.8.203.124:40249)
0811 03:00:02.828189 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.828210 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.828246 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:00:02.828866 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.828883 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.829198 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.829551 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.830325 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.831369 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.832306 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.832883 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.833001 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.835672 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.835678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.835688 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.835710 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.837244 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.837757 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.837763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.837769 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.837913 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.838215 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.838217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.838218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.838597 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.838777 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.839059 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.839351 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":124,"threads_started":1}
I20260811 03:00:02.840242 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.831163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:36479 (local address 127.0.0.1:41334)
0811 03:00:02.831596 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.831635 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.831748 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.832004 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.832010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.832368 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.832514 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.832520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.833368 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.833371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.834073 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.834083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.834676 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.834680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.834781 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.835307 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.835327 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.836825 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.838593 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.838602 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.838607 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.838901 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.839199 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.839202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.839204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.839291 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.839673 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.839678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.839792 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.839942 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.840100 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":86,"threads_started":1}
I20260811 03:00:02.842125 3473 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:00:02.843712 3318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } attempt: 3
I20260811 03:00:02.846374 3160 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } attempt: 3
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
I20260811 03:00:02.852633 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.842756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.8.203.125:33985)
0811 03:00:02.842885 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.842889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.842902 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:00:02.842941 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.842944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.842987 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.843062 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.843790 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.844450 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.845383 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.845702 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.845787 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.848273 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.848276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.848279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.848305 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.849879 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.850396 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.850400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.850401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.850489 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.851069 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.851072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.851074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.851356 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.851453 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.852365 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.852477 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 03:00:02.853427 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.844081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56294)
0811 03:00:02.844187 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.844201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.844272 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.844505 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.844508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.844678 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.844778 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.844785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.845660 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.845663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.846392 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.846404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.846967 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.846970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.847060 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.848474 (+ 1414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.848487 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.849861 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.851873 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.851877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.851879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.852120 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.852408 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.852411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.852413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.852496 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.852936 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.852940 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.853027 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.853119 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.853235 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 03:00:02.853452 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.842570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:33985 (local address 127.0.0.1:36070)
0811 03:00:02.842706 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.842719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.842784 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.843088 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.843091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.843214 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.843645 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.843654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.844566 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.844569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.845259 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.845269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.846025 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.846030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.846105 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.846539 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.846552 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.848136 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.849985 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.849988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.849990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.850269 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.850860 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.850863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.850865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.850951 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.852043 (+ 1092us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.852047 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.852149 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.853016 (+ 867us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.853281 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":1}
I20260811 03:00:02.853807 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.842590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.8.203.126:36479)
0811 03:00:02.842691 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.842695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.842773 (+ 78us) server_negotiation.cc:407] Connection header received
0811 03:00:02.842893 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.842895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.842929 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.842983 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.843585 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.844224 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.845113 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.845449 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.845533 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.849945 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.849949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.849951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.849974 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.851514 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.851913 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.851916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.851918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.851997 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.852315 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.852317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.852319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.852513 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.852716 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.853557 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.853665 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 03:00:02.853904 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.844237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56294 (local address 127.8.203.124:40249)
0811 03:00:02.844332 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.844335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.844346 (+ 11us) server_negotiation.cc:407] Connection header received
0811 03:00:02.844374 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.844376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.844412 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.844482 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.844887 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.845569 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.846503 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.846820 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.846954 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.849984 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.849988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.849990 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.850024 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.851767 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.852225 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.852228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.852230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.852309 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.852598 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.852602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.852603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.852806 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.852891 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.853636 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.853777 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":39,"spinlock_wait_cycles":6912}
I20260811 03:00:02.854452 3473 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:00:02.856966 3166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.846723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56310)
0811 03:00:02.846815 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.846829 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.846896 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.847712 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.847716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.847890 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.847988 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.847995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.848925 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.848929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.849640 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.849651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.850605 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.850610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.850698 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.851528 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.851543 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.852852 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.855468 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.855472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.855473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.855691 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.856008 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.856012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.856014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.856115 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.856546 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.856549 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.856637 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.856716 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.856830 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 03:00:02.857498 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.846928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.8.203.124:40249)
0811 03:00:02.847518 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.847525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.847536 (+ 11us) server_negotiation.cc:407] Connection header received
0811 03:00:02.847562 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.847564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.847594 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.847659 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.848108 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.848815 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.849752 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.850081 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.850887 (+ 806us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.852991 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.852995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.852998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.853022 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.855358 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.855804 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.855807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.855808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.855890 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.856222 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.856225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.856227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.856428 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.857129 (+ 701us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.857217 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.857346 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":159,"threads_started":1}
I20260811 03:00:02.857635 3318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: Bootstrap starting.
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
I20260811 03:00:02.860504 3318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:02.860631 3160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: Bootstrap starting.
I20260811 03:00:02.861472 3318 log.cc:824] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:00:02 dist-test-slave-gjw7 krb5kdc[3008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417202, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
I20260811 03:00:02.863734 3318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: No bootstrap required, opened a new log
I20260811 03:00:02.864312 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.855240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36086 (local address 127.8.203.125:33985)
0811 03:00:02.855337 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.855341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.855487 (+ 146us) server_negotiation.cc:407] Connection header received
0811 03:00:02.855587 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.855591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.855631 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.855694 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.856411 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.856872 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.857839 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.858135 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.858201 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.860889 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.860892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.860895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.860919 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.862364 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.862844 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.862848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.862849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.862932 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.863241 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.863245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.863246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.863470 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.863644 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.864070 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.864176 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 03:00:02.864436 3160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:02.865257 3160 log.cc:824] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:02.866098 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.855088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.8.203.124:40249)
0811 03:00:02.857409 (+ 2321us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.857413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.857426 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:00:02.857456 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.857459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.857494 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.857776 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.858211 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.858853 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.859881 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.860195 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.860260 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.862700 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.862704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.862706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.862728 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.864244 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.864867 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.864870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.864871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.864952 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.865217 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.865220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.865221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.865504 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.865576 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.865875 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.865974 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2250}
I20260811 03:00:02.866662 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.854906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:40249 (local address 127.0.0.1:56318)
0811 03:00:02.855652 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:02.855664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:02.855719 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:02.857882 (+ 2163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:02.857885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:02.858036 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:02.858106 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.858112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.858978 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.858981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.859754 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:02.859766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.860453 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.860456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:02.860531 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.861129 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:02.861140 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:02.862582 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:02.864342 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.864345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.864347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.864764 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.865047 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:02.865049 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:02.865051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:02.865132 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:02.865630 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:02.865633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:02.865714 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 03:00:02.866417 (+ 703us) client_negotiation.cc:240] Negotiation successful
0811 03:00:02.866535 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683}
I20260811 03:00:02.867306 3160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: No bootstrap required, opened a new log
I20260811 03:00:02.867647 3318 raft_consensus.cc:348] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:02.867882 3318 raft_consensus.cc:374] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:02.867936 3318 raft_consensus.cc:729] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e67af106d5d841df8aa505522513485a, State: Initialized, Role: FOLLOWER
I20260811 03:00:02.868397 3318 consensus_queue.cc:260] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:02.868870 3489 sys_catalog.cc:455] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:02.869071 3489 sys_catalog.cc:458] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:00:02.869084 3473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821: Bootstrap starting.
I20260811 03:00:02.870304 3318 sys_catalog.cc:564] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:02.870836 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:00:02.871224 3160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:02.871441 3160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:02.871495 3160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d785a1a1c1c41b391cdd7f3c8ff0d71, State: Initialized, Role: FOLLOWER
I20260811 03:00:02.871563 3473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:02.871984 3160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:02.872674 3473 log.cc:824] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:02.873384 3160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:02.874943 3473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821: No bootstrap required, opened a new log
W20260811 03:00:02.876638 3513 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:00:02.877264 3513 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:00:02.873927 3176 master_runner.cc:186] Error getting master registration for 127.8.203.126:36479: OK, instance_id { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" instance_seqno: 1786417202386128 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:00:02.877671 3176 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 03:00:02.878332 3545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:00:02.878405 3545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:00:02.878768 3473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:02.879030 3473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:02.879112 3473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14be8ff9ce1d40b3ae10df81c234e821, State: Initialized, Role: FOLLOWER
I20260811 03:00:02.877152 3514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:02.879725 3514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:00:02.881534 3473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
W20260811 03:00:02.882627 3069 tablet.cc:2369] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:00:02.883162 3473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:02.883509 3029 master_runner.cc:186] Error getting master registration for 127.8.203.124:40249: OK, instance_id { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" instance_seqno: 1786417202819833 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:00:02.886821 3029 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:00:02.888670 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.878181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41354 (local address 127.8.203.126:36479)
0811 03:00:02.878287 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.878291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.880023 (+ 1732us) server_negotiation.cc:407] Connection header received
0811 03:00:02.880191 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.880195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.880241 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.880290 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.881233 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.882011 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.883136 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.883475 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.883551 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.885350 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.885355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.885358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.885383 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.886958 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.887458 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.887462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.887464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.887550 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.887822 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.887825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.887827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.888037 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.888165 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.888402 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.888515 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:00:02.893044 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:02.893465 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 03:00:02.896158 3575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:00:02.896238 3575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:00:02.897512 3327 master_runner.cc:263] No leader master found from master 14be8ff9ce1d40b3ae10df81c234e821
I20260811 03:00:02.897579 3327 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:00:02.899281 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.878596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36092 (local address 127.8.203.125:33985)
0811 03:00:02.878697 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.878700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.888713 (+ 10013us) server_negotiation.cc:407] Connection header received
0811 03:00:02.889311 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.889315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.889374 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.889427 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.890267 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.890991 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.892060 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.892371 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.892446 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.894188 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.894193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.894196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.894222 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.897372 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.897903 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.897906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.897908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.897994 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.898285 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.898288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.898290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.898526 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.898649 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.899011 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.899127 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 03:00:02.905974 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:02.879287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56322 (local address 127.8.203.124:40249)
0811 03:00:02.879389 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:02.879393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:02.897865 (+ 18472us) server_negotiation.cc:407] Connection header received
0811 03:00:02.898124 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:02.898128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:02.898175 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:02.898226 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:02.898996 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.899703 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.900697 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:02.900964 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:02.901022 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:02.902581 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:02.902584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:02.902587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:02.902613 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:02.904135 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.904605 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.904609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.904610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.904693 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:02.904990 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:02.904993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:02.904995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:02.905193 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:02.905328 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:02.905678 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 03:00:02.905791 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 03:00:03.066357 3371 tablet.cc:2369] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:00:03.088392 3211 tablet.cc:2369] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:00:03.221518 3561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:03.221757 3561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:03.222813 3561 leader_election.cc:288] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d785a1a1c1c41b391cdd7f3c8ff0d71 (127.8.203.126:36479), e67af106d5d841df8aa505522513485a (127.8.203.125:33985)
I20260811 03:00:03.223263 3257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14be8ff9ce1d40b3ae10df81c234e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e67af106d5d841df8aa505522513485a" is_pre_election: true
I20260811 03:00:03.223268 3114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14be8ff9ce1d40b3ae10df81c234e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" is_pre_election: true
I20260811 03:00:03.223546 3257 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14be8ff9ce1d40b3ae10df81c234e821 in term 0.
I20260811 03:00:03.223563 3114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14be8ff9ce1d40b3ae10df81c234e821 in term 0.
I20260811 03:00:03.223973 3368 leader_election.cc:302] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14be8ff9ce1d40b3ae10df81c234e821, 2d785a1a1c1c41b391cdd7f3c8ff0d71, e67af106d5d841df8aa505522513485a; no voters:
I20260811 03:00:03.224236 3561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:00:03.224315 3561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:00:03.224344 3561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:03.225380 3561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:03.225674 3561 leader_election.cc:288] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [CANDIDATE]: Term 1 election: Requested vote from peers 2d785a1a1c1c41b391cdd7f3c8ff0d71 (127.8.203.126:36479), e67af106d5d841df8aa505522513485a (127.8.203.125:33985)
I20260811 03:00:03.225997 3114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14be8ff9ce1d40b3ae10df81c234e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71"
I20260811 03:00:03.226022 3257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14be8ff9ce1d40b3ae10df81c234e821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e67af106d5d841df8aa505522513485a"
I20260811 03:00:03.226119 3114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:03.226115 3257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:03.227099 3257 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14be8ff9ce1d40b3ae10df81c234e821 in term 1.
I20260811 03:00:03.227169 3114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14be8ff9ce1d40b3ae10df81c234e821 in term 1.
I20260811 03:00:03.227391 3368 leader_election.cc:302] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14be8ff9ce1d40b3ae10df81c234e821, e67af106d5d841df8aa505522513485a; no voters:
I20260811 03:00:03.227571 3561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:03.227787 3561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [term 1 LEADER]: Becoming Leader. State: Replica: 14be8ff9ce1d40b3ae10df81c234e821, State: Running, Role: LEADER
I20260811 03:00:03.228084 3561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } }
I20260811 03:00:03.229871 3581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14be8ff9ce1d40b3ae10df81c234e821. Latest consensus state: current_term: 1 leader_uuid: "14be8ff9ce1d40b3ae10df81c234e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:03.229970 3581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:03.230311 3586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:00:03.231151 3586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:00:03.235744 3114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [term 1 FOLLOWER]: Refusing update from remote peer 14be8ff9ce1d40b3ae10df81c234e821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:03.235777 3257 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [term 1 FOLLOWER]: Refusing update from remote peer 14be8ff9ce1d40b3ae10df81c234e821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:03.236399 3581 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:03.236570 3561 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:03.240711 3514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14be8ff9ce1d40b3ae10df81c234e821. Latest consensus state: current_term: 1 leader_uuid: "14be8ff9ce1d40b3ae10df81c234e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:03.240711 3489 sys_catalog.cc:455] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14be8ff9ce1d40b3ae10df81c234e821. Latest consensus state: current_term: 1 leader_uuid: "14be8ff9ce1d40b3ae10df81c234e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:03.240830 3489 sys_catalog.cc:458] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:00:03.240830 3514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:00:03.242650 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14be8ff9ce1d40b3ae10df81c234e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:03.242762 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:03.243530 3514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14be8ff9ce1d40b3ae10df81c234e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:03.243530 3489 sys_catalog.cc:455] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14be8ff9ce1d40b3ae10df81c234e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:03.243639 3514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:00:03.243650 3489 sys_catalog.cc:458] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:00:03.244211 3586 catalog_manager.cc:1347] Generated new cluster ID: da60db355ac34a32982ceb061f15c4f8
I20260811 03:00:03.244273 3586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:00:03.244253 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14be8ff9ce1d40b3ae10df81c234e821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d785a1a1c1c41b391cdd7f3c8ff0d71" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36479 } } peers { permanent_uuid: "e67af106d5d841df8aa505522513485a" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 33985 } } peers { permanent_uuid: "14be8ff9ce1d40b3ae10df81c234e821" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 40249 } } }
I20260811 03:00:03.244354 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:03.261081 3586 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:00:03.262140 3586 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:00:03.276328 3586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14be8ff9ce1d40b3ae10df81c234e821: Generated new TSK 0
I20260811 03:00:03.277230 3586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:00:03.878726 3513 catalog_manager.cc:1259] Loaded cluster ID: da60db355ac34a32982ceb061f15c4f8
I20260811 03:00:03.878803 3513 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: loading cluster ID for follower catalog manager: success
I20260811 03:00:03.879741 3545 catalog_manager.cc:1259] Loaded cluster ID: da60db355ac34a32982ceb061f15c4f8
I20260811 03:00:03.879807 3545 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: loading cluster ID for follower catalog manager: success
I20260811 03:00:03.881096 3513 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: acquiring CA information for follower catalog manager: success
I20260811 03:00:03.881999 3545 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: acquiring CA information for follower catalog manager: success
I20260811 03:00:03.882607 3513 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e67af106d5d841df8aa505522513485a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:00:03.883258 3545 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d785a1a1c1c41b391cdd7f3c8ff0d71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:00:04.353358 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 3029
I20260811 03:00:04.373996 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 3176
I20260811 03:00:04.390462 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 3327
2026-08-11T03:00:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3520 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:00:04 dist-test-slave-gjw7 krb5kdc[3608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:00:04 dist-test-slave-gjw7 krb5kdc[3608](info): set up 2 sockets
Aug 11 03:00:04 dist-test-slave-gjw7 krb5kdc[3608](info): commencing operation
krb5kdc: starting...
W20260811 03:00:05.596858 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:00:05 dist-test-slave-gjw7 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:00:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:00:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:00:05.719995 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:35871
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:35871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:05.831377 3625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:05.831629 3625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:05.831668 3625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:05.835539 3625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:05.835587 3625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:05.835604 3625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:05.835619 3625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:05.835638 3625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:05.840749 3625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:35871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:35871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:05.841748 3625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:05.842694 3625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:05.850901 3639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:05.850874 3644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:05.850919 3637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:05.851011 3625 server_base.cc:1034] running on GCE node
I20260811 03:00:05.851727 3625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:05.852751 3625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:05.853902 3625 hybrid_clock.cc:648] HybridClock initialized: now 1786417205853879 us; error 34 us; skew 500 ppm
Aug 11 03:00:05 dist-test-slave-gjw7 krb5kdc[3608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417205, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:05.856463 3625 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:00:05.857280 3625 webserver.cc:460] Webserver started at http://127.8.203.126:34659/ using document root <none> and password file <none>
I20260811 03:00:05.857841 3625 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:05.857903 3625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:05.858104 3625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:05.860081 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "318a078e74ed469ebd7c684903fa97b9"
format_stamp: "Formatted at 2026-08-11 03:00:05 on dist-test-slave-gjw7"
server_key: "ea522c25e3388bb2642875e80931dba0"
server_key_iv: "66d8cd21b7e6c4bf299f60e474b62868"
server_key_version: "encryptionkey@0"
I20260811 03:00:05.860630 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "318a078e74ed469ebd7c684903fa97b9"
format_stamp: "Formatted at 2026-08-11 03:00:05 on dist-test-slave-gjw7"
server_key: "ea522c25e3388bb2642875e80931dba0"
server_key_iv: "66d8cd21b7e6c4bf299f60e474b62868"
server_key_version: "encryptionkey@0"
I20260811 03:00:05.864190 3625 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 03:00:05.866508 3656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:05.867648 3625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:00:05.867761 3625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "318a078e74ed469ebd7c684903fa97b9"
format_stamp: "Formatted at 2026-08-11 03:00:05 on dist-test-slave-gjw7"
server_key: "ea522c25e3388bb2642875e80931dba0"
server_key_iv: "66d8cd21b7e6c4bf299f60e474b62868"
server_key_version: "encryptionkey@0"
I20260811 03:00:05.867856 3625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:05.883541 3625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:05.886729 3625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:05.886909 3625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:05.893539 3625 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:35871
I20260811 03:00:05.893567 3751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:35871 every 8 connection(s)
I20260811 03:00:05.894491 3625 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:00:05.896031 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 3625
I20260811 03:00:05.896126 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:00:05.896386 9005 external_mini_cluster.cc:1428] Setting key c078060fc912a1984e025fc2231bf18a
I20260811 03:00:05.897362 3753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:05.902926 3753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9: Bootstrap starting.
Aug 11 03:00:05 dist-test-slave-gjw7 krb5kdc[3608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:05.905488 3753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:05.906303 3753 log.cc:824] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:05.908452 3753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9: No bootstrap required, opened a new log
I20260811 03:00:05.910198 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:05.897754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.8.203.126:35871)
0811 03:00:05.898204 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:05.898220 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:05.898256 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:00:05.898909 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:05.898926 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:05.899243 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:05.899553 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:05.900341 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:05.901355 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:05.902192 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:05.902675 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:05.902823 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:05.905416 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:05.905422 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:05.905436 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:05.905463 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:05.907308 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:05.907771 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:05.907775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:05.907779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:05.907888 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:05.908147 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:05.908149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:05.908150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:05.908541 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:05.908703 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:05.908969 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 03:00:05.909232 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":141,"threads_started":1}
I20260811 03:00:05.911830 3753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318a078e74ed469ebd7c684903fa97b9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35871 } }
I20260811 03:00:05.912098 3753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:05.912195 3753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 318a078e74ed469ebd7c684903fa97b9, State: Initialized, Role: FOLLOWER
I20260811 03:00:05.912654 3753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318a078e74ed469ebd7c684903fa97b9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35871 } }
I20260811 03:00:05.912797 3753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:00:05.912863 3753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:00:05.912966 3753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:05.913038 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:00:05.913935 3753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318a078e74ed469ebd7c684903fa97b9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35871 } }
I20260811 03:00:05.914290 3753 leader_election.cc:302] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 318a078e74ed469ebd7c684903fa97b9; no voters:
I20260811 03:00:05.914651 3753 leader_election.cc:288] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:00:05.914733 3760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:05.914945 3760 raft_consensus.cc:686] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [term 1 LEADER]: Becoming Leader. State: Replica: 318a078e74ed469ebd7c684903fa97b9, State: Running, Role: LEADER
I20260811 03:00:05.915285 3760 consensus_queue.cc:237] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318a078e74ed469ebd7c684903fa97b9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35871 } }
I20260811 03:00:05.915773 3753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:05.916780 3768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 318a078e74ed469ebd7c684903fa97b9. Latest consensus state: current_term: 1 leader_uuid: "318a078e74ed469ebd7c684903fa97b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318a078e74ed469ebd7c684903fa97b9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35871 } } }
I20260811 03:00:05.917135 3768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:05.917039 3767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "318a078e74ed469ebd7c684903fa97b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "318a078e74ed469ebd7c684903fa97b9" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 35871 } } }
I20260811 03:00:05.917466 3767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:05.918028 3778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:00:05.920491 3778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:00:05.925966 3778 catalog_manager.cc:1347] Generated new cluster ID: 69914cee34b44cf68f5721da1d8d56de
I20260811 03:00:05.926016 3778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:00:05.962321 3778 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:00:05.963279 3778 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:00:05.970781 3778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 318a078e74ed469ebd7c684903fa97b9: Generated new TSK 0
I20260811 03:00:05.971486 3778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:00:06.019989 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 3625
2026-08-11T03:00:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1630 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:00:06 dist-test-slave-gjw7 krb5kdc[3811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:00:06 dist-test-slave-gjw7 krb5kdc[3811](info): set up 2 sockets
Aug 11 03:00:06 dist-test-slave-gjw7 krb5kdc[3811](info): commencing operation
krb5kdc: starting...
W20260811 03:00:07.245754 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:00:07 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:00:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:00:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:00:07.365832 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:40397
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:07.475733 3832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:07.476020 3832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:07.476078 3832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:07.480007 3832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:07.480062 3832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:07.480088 3832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:07.480109 3832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:07.480134 3832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:07.485306 3832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:07.486326 3832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:07.487310 3832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:07.495491 3842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:07.495488 3840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:07.495512 3844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:07.495810 3832 server_base.cc:1034] running on GCE node
I20260811 03:00:07.496368 3832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:07.497462 3832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:07.498617 3832 hybrid_clock.cc:648] HybridClock initialized: now 1786417207498596 us; error 30 us; skew 500 ppm
Aug 11 03:00:07 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417207, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:07.501200 3832 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:00:07.502043 3832 webserver.cc:460] Webserver started at http://127.8.203.126:41049/ using document root <none> and password file <none>
I20260811 03:00:07.502578 3832 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:07.502642 3832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:07.502844 3832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:07.505095 3832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "ac4b9b0b201340a98d84cb8e9bd304c2"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "e8c7c24cd714c5d5d28a251a5fb94d11"
server_key_iv: "266b6df4f80f792c01e76857783b82da"
server_key_version: "encryptionkey@0"
I20260811 03:00:07.505668 3832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "ac4b9b0b201340a98d84cb8e9bd304c2"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "e8c7c24cd714c5d5d28a251a5fb94d11"
server_key_iv: "266b6df4f80f792c01e76857783b82da"
server_key_version: "encryptionkey@0"
I20260811 03:00:07.509191 3832 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:00:07.511508 3855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:07.512634 3832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:07.512746 3832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "ac4b9b0b201340a98d84cb8e9bd304c2"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "e8c7c24cd714c5d5d28a251a5fb94d11"
server_key_iv: "266b6df4f80f792c01e76857783b82da"
server_key_version: "encryptionkey@0"
I20260811 03:00:07.512837 3832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:07.554980 3832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:07.557827 3832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:07.558013 3832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:07.564736 3832 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:40397
I20260811 03:00:07.565004 3974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:40397 every 8 connection(s)
I20260811 03:00:07.565719 3832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:00:07.568423 3976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:07.571486 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 3832
I20260811 03:00:07.571595 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:00:07.571844 9005 external_mini_cluster.cc:1428] Setting key c2ede866fd3eeffff8a00f307593673b
I20260811 03:00:07.574056 3976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Bootstrap starting.
I20260811 03:00:07.576442 3976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:07.577257 3976 log.cc:824] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:07.579227 3976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: No bootstrap required, opened a new log
Aug 11 03:00:07 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:07.582644 3976 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:07.582881 3976 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:07.582931 3976 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4b9b0b201340a98d84cb8e9bd304c2, State: Initialized, Role: FOLLOWER
I20260811 03:00:07.583350 3976 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:07.583513 3976 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:00:07.583581 3976 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:00:07.583683 3976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:07.584589 3976 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:07.584961 3976 leader_election.cc:302] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac4b9b0b201340a98d84cb8e9bd304c2; no voters:
I20260811 03:00:07.585368 3976 leader_election.cc:288] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:00:07.585441 3987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:07.585634 3987 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 LEADER]: Becoming Leader. State: Replica: ac4b9b0b201340a98d84cb8e9bd304c2, State: Running, Role: LEADER
I20260811 03:00:07.586302 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:07.573154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.8.203.126:40397)
0811 03:00:07.573575 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:07.573590 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:07.573627 (+ 37us) server_negotiation.cc:407] Connection header received
0811 03:00:07.574262 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:07.574280 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:07.574611 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:07.574879 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:07.575724 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:07.576725 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:07.577837 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:07.578325 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:07.578486 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:07.581096 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:07.581101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:07.581114 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:07.581140 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:07.583271 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:07.583745 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:07.583752 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:07.583758 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:07.583886 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:07.584156 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:07.584158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:07.584159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:07.584558 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:07.584711 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:07.584986 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 03:00:07.585250 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":147,"threads_started":1}
I20260811 03:00:07.586553 3976 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:07.586575 3987 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:07.588958 3989 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } } }
I20260811 03:00:07.589078 3989 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:07.589224 3990 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac4b9b0b201340a98d84cb8e9bd304c2. Latest consensus state: current_term: 1 leader_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } } }
I20260811 03:00:07.589375 3990 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: This master's current role is: LEADER
W20260811 03:00:07.591559 4007 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:00:07.591622 4007 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:00:07.591691 4009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:00:07.592501 4009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:00:07.597328 4009 catalog_manager.cc:1347] Generated new cluster ID: 6b11a1f226274db0adbdf251ba802854
I20260811 03:00:07.597393 4009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:00:07.603916 4009 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:00:07.605041 4009 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:00:07.625509 4009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Generated new TSK 0
I20260811 03:00:07.626320 4009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 03:00:07.659502 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:40397
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:07.771438 4022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:07.771713 4022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:07.771776 4022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:07.775780 4022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:07.775842 4022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:07.775934 4022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:00:07.781092 4022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:07.782164 4022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:07.783160 4022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:07.792730 4042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:07.792730 4040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:07.793095 4039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:07.792989 4022 server_base.cc:1034] running on GCE node
I20260811 03:00:07.793643 4022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:07.794255 4022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:07.795425 4022 hybrid_clock.cc:648] HybridClock initialized: now 1786417207795408 us; error 30 us; skew 500 ppm
Aug 11 03:00:07 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417207, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:07.798071 4022 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:00:07.798892 4022 webserver.cc:460] Webserver started at http://127.8.203.65:44673/ using document root <none> and password file <none>
I20260811 03:00:07.799418 4022 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:07.799484 4022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:07.799685 4022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:07.801754 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "3313b669515a433387062842ed25d59a"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "f158c8107e5bc1da439f51edd09a74f0"
server_key_iv: "3f81cf1b5016818614bbb8659ae9872e"
server_key_version: "encryptionkey@0"
I20260811 03:00:07.802274 4022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "3313b669515a433387062842ed25d59a"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "f158c8107e5bc1da439f51edd09a74f0"
server_key_iv: "3f81cf1b5016818614bbb8659ae9872e"
server_key_version: "encryptionkey@0"
I20260811 03:00:07.805850 4022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:00:07.808115 4053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:07.809257 4022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:00:07.809409 4022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "3313b669515a433387062842ed25d59a"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "f158c8107e5bc1da439f51edd09a74f0"
server_key_iv: "3f81cf1b5016818614bbb8659ae9872e"
server_key_version: "encryptionkey@0"
I20260811 03:00:07.809501 4022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:07.832163 4022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:07.835386 4022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:07.835592 4022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:07.836450 4022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:07.837386 4022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:07.837435 4022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:07.837488 4022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:07.837517 4022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:07.846053 4022 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:34913
I20260811 03:00:07.846073 4283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:34913 every 8 connection(s)
I20260811 03:00:07.846993 4022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
Aug 11 03:00:07 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417207, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:07.856449 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 4022
I20260811 03:00:07.856562 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:00:07.856853 9005 external_mini_cluster.cc:1428] Setting key db72e23a5471ebf069b57bc7fab05eda
I20260811 03:00:07.860291 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:07.848787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:51307 (local address 127.8.203.126:40397)
0811 03:00:07.848927 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:07.848932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:07.849824 (+ 892us) server_negotiation.cc:407] Connection header received
0811 03:00:07.851122 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:07.851125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:07.851171 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:07.851240 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:07.852360 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:07.852908 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:07.854071 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:07.854320 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:07.854394 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:07.856793 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:07.856797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:07.856800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:07.856829 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:07.858348 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:07.858814 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:07.858818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:07.858820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:07.858902 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:07.859207 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:07.859210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:07.859212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:07.859398 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:07.859482 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:07.859987 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 03:00:07.860117 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 03:00:07.861086 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:07.849189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:51307)
0811 03:00:07.849667 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:07.849711 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:07.850930 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:07.851400 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:07.851406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:07.851768 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:07.852243 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:07.852259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:07.852997 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:07.853000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:07.853957 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:07.853967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:07.854453 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:07.854456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:07.854601 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:07.855114 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:07.855130 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:07.856606 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:07.858450 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:07.858456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:07.858471 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:07.858714 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:07.858981 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:07.858984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:07.858986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:07.859120 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:07.859507 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:07.859513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:07.859762 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 03:00:07.859928 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 03:00:07.860252 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":121,"threads_started":1}
I20260811 03:00:07.862308 4286 heartbeater.cc:347] Connected to a master server at 127.8.203.126:40397
I20260811 03:00:07.862529 4286 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:07.863013 4286 heartbeater.cc:511] Master 127.8.203.126:40397 requested a full tablet report, sending...
I20260811 03:00:07.864660 3903 ts_manager.cc:194] Registered new tserver with Master: 3313b669515a433387062842ed25d59a (127.8.203.65:34913)
I20260811 03:00:07.866223 3903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:51307
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 03:00:07.884033 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:40397
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:07.996029 4291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:07.996285 4291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:07.996343 4291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:08.000300 4291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:08.000350 4291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:08.000428 4291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:00:08.005499 4291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:08.006479 4291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:08.007442 4291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:08.016778 4304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:08.016767 4302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:08.016799 4309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:08.017530 4291 server_base.cc:1034] running on GCE node
I20260811 03:00:08.017922 4291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:08.018503 4291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:08.019654 4291 hybrid_clock.cc:648] HybridClock initialized: now 1786417208019636 us; error 28 us; skew 500 ppm
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:08.022342 4291 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:00:08.023185 4291 webserver.cc:460] Webserver started at http://127.8.203.66:33989/ using document root <none> and password file <none>
I20260811 03:00:08.023730 4291 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:08.023789 4291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:08.023991 4291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:08.026212 4291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "b9bda3783ad8432c8369da6f0bd10a5e"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "7fa1ec02984b0edbcb82f84dd3d595cf"
server_key_iv: "7623275199282cab8c8d279b04ea8306"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.026782 4291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "b9bda3783ad8432c8369da6f0bd10a5e"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "7fa1ec02984b0edbcb82f84dd3d595cf"
server_key_iv: "7623275199282cab8c8d279b04ea8306"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.030464 4291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 03:00:08.032843 4322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.034044 4291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 03:00:08.034140 4291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "b9bda3783ad8432c8369da6f0bd10a5e"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "7fa1ec02984b0edbcb82f84dd3d595cf"
server_key_iv: "7623275199282cab8c8d279b04ea8306"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.034225 4291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:08.044330 4291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:08.047354 4291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:08.047504 4291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:08.048017 4291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:08.048821 4291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:08.048869 4291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.048918 4291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:08.048936 4291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.057401 4291 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:43817
I20260811 03:00:08.057433 4559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:43817 every 8 connection(s)
I20260811 03:00:08.058238 4291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:00:08.061235 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 4291
I20260811 03:00:08.061373 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:00:08.061626 9005 external_mini_cluster.cc:1428] Setting key 558bc628b26124f1e1a8d267f9ffbfe5
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:08.071674 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.059956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:57909 (local address 127.8.203.126:40397)
0811 03:00:08.060087 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.060092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.060878 (+ 786us) server_negotiation.cc:407] Connection header received
0811 03:00:08.062248 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.062251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.062311 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.062387 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:08.063548 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.064065 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.065185 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.065461 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.065570 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.068080 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:08.068083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:08.068085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:08.068109 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:08.069661 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.070163 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.070167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.070169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.070261 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.070555 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.070558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.070560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.070792 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:08.070910 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.071365 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.071497 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 03:00:08.072432 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.060284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:57909)
0811 03:00:08.060726 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.060768 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.062031 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.062570 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.062577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.062914 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:08.063397 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.063411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.064183 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.064186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.065062 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.065072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.065611 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.065615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.065790 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.066321 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:08.066347 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:08.067922 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:08.069789 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.069795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.069807 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.070062 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.070348 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.070350 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.070351 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.070475 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.070893 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:08.070898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:08.071131 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.071286 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.071550 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
I20260811 03:00:08.073632 4560 heartbeater.cc:347] Connected to a master server at 127.8.203.126:40397
I20260811 03:00:08.073871 4560 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:08.074436 4560 heartbeater.cc:511] Master 127.8.203.126:40397 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
I20260811 03:00:08.075590 3903 ts_manager.cc:194] Registered new tserver with Master: b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
I20260811 03:00:08.076160 3903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:57909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 03:00:08.089340 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:40397
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:08.200194 4575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:08.200428 4575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:08.200482 4575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:08.204344 4575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:08.204403 4575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:08.204488 4575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:00:08.209518 4575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:08.210448 4575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:08.211409 4575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:08.220975 4586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:08.221022 4595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:08.220985 4587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:08.221338 4575 server_base.cc:1034] running on GCE node
I20260811 03:00:08.221697 4575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:08.222245 4575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:08.223397 4575 hybrid_clock.cc:648] HybridClock initialized: now 1786417208223371 us; error 37 us; skew 500 ppm
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:08.226006 4575 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:00:08.226784 4575 webserver.cc:460] Webserver started at http://127.8.203.67:39857/ using document root <none> and password file <none>
I20260811 03:00:08.227288 4575 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:08.227346 4575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:08.227514 4575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:08.229496 4575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "39fa89d23aef43ad87a152b487d0abda"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "67e2d007ec72afbfb57b2df659ab77ee"
server_key_iv: "ba3a9714d4ec136dd8d5579877850692"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.230008 4575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "39fa89d23aef43ad87a152b487d0abda"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "67e2d007ec72afbfb57b2df659ab77ee"
server_key_iv: "ba3a9714d4ec136dd8d5579877850692"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.233507 4575 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:00:08.235739 4603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.236905 4575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:08.237013 4575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "39fa89d23aef43ad87a152b487d0abda"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "67e2d007ec72afbfb57b2df659ab77ee"
server_key_iv: "ba3a9714d4ec136dd8d5579877850692"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.237097 4575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:08.250978 4575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:08.253598 4575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:08.253739 4575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:08.254238 4575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:08.255020 4575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:08.255065 4575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.255102 4575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:08.255122 4575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.263387 4575 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43557
I20260811 03:00:08.263414 4821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43557 every 8 connection(s)
I20260811 03:00:08.264226 4575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:00:08.265443 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 4575
I20260811 03:00:08.265552 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:00:08.265807 9005 external_mini_cluster.cc:1428] Setting key 4dc8fa2dc65885959f5107dc73815dc4
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:08.276460 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.266010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:49445 (local address 127.8.203.126:40397)
0811 03:00:08.266171 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.266176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.267098 (+ 922us) server_negotiation.cc:407] Connection header received
0811 03:00:08.268570 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.268573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.268619 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.268682 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:08.269843 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.270353 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.271384 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.271581 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.271644 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.273804 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:08.273807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:08.273809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:08.273833 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:08.274900 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.275313 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.275315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.275317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.275392 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.275626 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.275628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.275629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.275779 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:08.275846 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.276244 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.276345 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:00:08.277185 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.266440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:49445)
0811 03:00:08.266969 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.267016 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.268372 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.268829 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.268848 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.269178 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:08.269721 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.269738 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.270445 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.270448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.271292 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.271301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.271680 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.271682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.271815 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.272280 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:08.272295 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:08.273656 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:08.274992 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.274996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.275008 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.275239 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.275451 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.275453 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.275455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.275566 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.275846 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:08.275850 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:08.276069 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.276215 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.276451 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260811 03:00:08.278203 4822 heartbeater.cc:347] Connected to a master server at 127.8.203.126:40397
I20260811 03:00:08.278419 4822 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:08.278947 4822 heartbeater.cc:511] Master 127.8.203.126:40397 requested a full tablet report, sending...
I20260811 03:00:08.279891 3903 ts_manager.cc:194] Registered new tserver with Master: 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557)
I20260811 03:00:08.280397 3903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:49445
I20260811 03:00:08.290822 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:00:08.298916 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.292693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56544 (local address 127.8.203.126:40397)
0811 03:00:08.292793 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.292797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.293033 (+ 236us) server_negotiation.cc:407] Connection header received
0811 03:00:08.293127 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.293129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.293169 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.293229 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:08.293879 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.294358 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.295137 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.295329 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.295392 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.296536 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:08.296539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:08.296546 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:08.296567 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:08.297679 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.298036 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.298038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.298039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.298101 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.298274 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.298276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.298277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.298465 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:08.298558 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.298704 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.298798 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 03:00:08.301273 3903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:00:08.304185 3903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:00:08.318303 4848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.312207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43817 (local address 127.0.0.1:51822)
0811 03:00:08.312973 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.312986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.313077 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.314035 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.314037 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.314054 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:08.314132 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.314137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.315188 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.315191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.316823 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.316834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.317735 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.317739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.317896 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.317941 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.318034 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.318150 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":127,"thread_start_us":94,"threads_started":1}
I20260811 03:00:08.318387 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.312210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.8.203.67:43557)
0811 03:00:08.313355 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.313367 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.313392 (+ 25us) server_negotiation.cc:407] Connection header received
0811 03:00:08.313470 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.313475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.313676 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.313836 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:08.314125 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.315230 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.316980 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.317887 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.317986 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.318125 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.318191 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.318265 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1011,"thread_start_us":87,"threads_started":1}
I20260811 03:00:08.318303 4849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.312091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43557 (local address 127.0.0.1:35746)
0811 03:00:08.312848 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.312908 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.313027 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.313842 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.313848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.313882 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:08.314005 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.314013 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.315354 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.315358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.316822 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.316834 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.317987 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.317991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.318071 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.318093 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.318140 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.318188 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":117,"thread_start_us":81,"threads_started":1}
I20260811 03:00:08.318881 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.313313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.8.203.65:34913)
0811 03:00:08.313675 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.313691 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.314216 (+ 525us) server_negotiation.cc:407] Connection header received
0811 03:00:08.314293 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.314298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.314479 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.314621 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:08.314836 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.315862 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.317057 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.317880 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.317986 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.318125 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.318653 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.318734 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":71,"threads_started":1}
I20260811 03:00:08.318780 4840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.312465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34913 (local address 127.0.0.1:41834)
0811 03:00:08.314035 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.314045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.314125 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.314615 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.314619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.314638 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:08.314726 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.314732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.316028 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.316031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.316926 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.316938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.318350 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.318354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.318434 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.318537 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.318605 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.318666 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1516,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260811 03:00:08.320216 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.312881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51822 (local address 127.8.203.66:43817)
0811 03:00:08.313522 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.313535 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.313554 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:00:08.313614 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.313618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.313752 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.313888 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:08.314232 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.315076 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.316967 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.317592 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.319789 (+ 2197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.319933 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.319982 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.320068 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":157,"threads_started":1}
I20260811 03:00:08.321607 4688 tablet_service.cc:1467] Processing CreateTablet for tablet 29edd748decb4594a1f4264560c973d0 (DEFAULT_TABLE table=test-table [id=d4ca294bf72e435483d68141dc1302bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:08.321655 4151 tablet_service.cc:1467] Processing CreateTablet for tablet 29edd748decb4594a1f4264560c973d0 (DEFAULT_TABLE table=test-table [id=d4ca294bf72e435483d68141dc1302bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:08.322566 4688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29edd748decb4594a1f4264560c973d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:08.322681 4151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29edd748decb4594a1f4264560c973d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:08.322682 4423 tablet_service.cc:1467] Processing CreateTablet for tablet 29edd748decb4594a1f4264560c973d0 (DEFAULT_TABLE table=test-table [id=d4ca294bf72e435483d68141dc1302bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:08.323575 4423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29edd748decb4594a1f4264560c973d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:08.328631 4858 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Bootstrap starting.
I20260811 03:00:08.329980 4865 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Bootstrap starting.
I20260811 03:00:08.330762 4861 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Bootstrap starting.
I20260811 03:00:08.330929 4858 tablet_bootstrap.cc:654] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:08.332233 4858 log.cc:824] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:08.332381 4861 tablet_bootstrap.cc:654] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:08.332540 4865 tablet_bootstrap.cc:654] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:08.333381 4865 log.cc:824] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:08.333590 4861 log.cc:824] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:08.334335 4858 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: No bootstrap required, opened a new log
I20260811 03:00:08.334527 4858 ts_tablet_manager.cc:1397] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:00:08.335230 4865 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: No bootstrap required, opened a new log
I20260811 03:00:08.335433 4861 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: No bootstrap required, opened a new log
I20260811 03:00:08.335425 4865 ts_tablet_manager.cc:1397] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:00:08.335597 4861 ts_tablet_manager.cc:1397] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 03:00:08.337415 4858 raft_consensus.cc:348] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.337641 4858 raft_consensus.cc:374] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:08.337692 4858 raft_consensus.cc:729] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39fa89d23aef43ad87a152b487d0abda, State: Initialized, Role: FOLLOWER
I20260811 03:00:08.338074 4858 consensus_queue.cc:260] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.338663 4822 heartbeater.cc:503] Master 127.8.203.126:40397 was elected leader, sending a full tablet report...
I20260811 03:00:08.339015 4858 ts_tablet_manager.cc:1428] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 03:00:08.339202 4861 raft_consensus.cc:348] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.339145 4865 raft_consensus.cc:348] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.339413 4861 raft_consensus.cc:374] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:08.339413 4865 raft_consensus.cc:374] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:08.339470 4861 raft_consensus.cc:729] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3313b669515a433387062842ed25d59a, State: Initialized, Role: FOLLOWER
I20260811 03:00:08.339480 4865 raft_consensus.cc:729] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9bda3783ad8432c8369da6f0bd10a5e, State: Initialized, Role: FOLLOWER
I20260811 03:00:08.339931 4861 consensus_queue.cc:260] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.339931 4865 consensus_queue.cc:260] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.340847 4560 heartbeater.cc:503] Master 127.8.203.126:40397 was elected leader, sending a full tablet report...
I20260811 03:00:08.340865 4861 ts_tablet_manager.cc:1428] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 03:00:08.341161 4286 heartbeater.cc:503] Master 127.8.203.126:40397 was elected leader, sending a full tablet report...
I20260811 03:00:08.342023 4865 ts_tablet_manager.cc:1428] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 03:00:08.345045 4869 raft_consensus.cc:482] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:08.345176 4869 raft_consensus.cc:504] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.346069 4869 leader_election.cc:288] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817), 3313b669515a433387062842ed25d59a (127.8.203.65:34913)
W20260811 03:00:08.348104 4287 tablet.cc:2369] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:00:08.349814 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.346243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43817 (local address 127.8.203.67:52427)
0811 03:00:08.346361 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.346377 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.346456 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.346795 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.346798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.346815 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:08.346969 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.346977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.347687 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.347690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.348699 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.348709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.349427 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.349431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.349533 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.349558 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.349625 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.349691 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":20}
I20260811 03:00:08.349817 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.346480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:52427 (local address 127.8.203.66:43817)
0811 03:00:08.346586 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.346590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.346604 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:00:08.346650 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.346654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.346693 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.346757 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:08.347073 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.347586 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.348813 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.349284 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.349403 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.349444 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.349640 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.349705 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:00:08.350064 4872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.346293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34913 (local address 127.8.203.67:56231)
0811 03:00:08.346581 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.346594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.346676 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.346911 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.346913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.346927 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:08.347041 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.347048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.347700 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.347704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.348933 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.348944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.349659 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.349662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.349756 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.349779 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.349860 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.349923 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260811 03:00:08.350066 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.346614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56231 (local address 127.8.203.65:34913)
0811 03:00:08.346722 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.346726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.346738 (+ 12us) server_negotiation.cc:407] Connection header received
0811 03:00:08.346781 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.346784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.346823 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.346892 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:08.347139 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.347604 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.349046 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.349537 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.349643 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.349686 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.349891 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.349963 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 03:00:08.350399 4462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29edd748decb4594a1f4264560c973d0" candidate_uuid: "39fa89d23aef43ad87a152b487d0abda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" is_pre_election: true
I20260811 03:00:08.350634 4462 raft_consensus.cc:2454] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39fa89d23aef43ad87a152b487d0abda in term 0.
I20260811 03:00:08.351047 4608 leader_election.cc:302] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e; no voters:
I20260811 03:00:08.351265 4869 raft_consensus.cc:2781] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:00:08.351249 4198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29edd748decb4594a1f4264560c973d0" candidate_uuid: "39fa89d23aef43ad87a152b487d0abda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3313b669515a433387062842ed25d59a" is_pre_election: true
I20260811 03:00:08.351344 4869 raft_consensus.cc:482] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:00:08.351377 4869 raft_consensus.cc:3037] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:08.351480 4198 raft_consensus.cc:2454] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39fa89d23aef43ad87a152b487d0abda in term 0.
I20260811 03:00:08.352402 4869 raft_consensus.cc:504] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.352715 4869 leader_election.cc:288] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [CANDIDATE]: Term 1 election: Requested vote from peers b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817), 3313b669515a433387062842ed25d59a (127.8.203.65:34913)
I20260811 03:00:08.353027 4462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29edd748decb4594a1f4264560c973d0" candidate_uuid: "39fa89d23aef43ad87a152b487d0abda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9bda3783ad8432c8369da6f0bd10a5e"
I20260811 03:00:08.353032 4198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29edd748decb4594a1f4264560c973d0" candidate_uuid: "39fa89d23aef43ad87a152b487d0abda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3313b669515a433387062842ed25d59a"
I20260811 03:00:08.353121 4462 raft_consensus.cc:3037] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:08.353125 4198 raft_consensus.cc:3037] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:08.354091 4462 raft_consensus.cc:2454] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39fa89d23aef43ad87a152b487d0abda in term 1.
I20260811 03:00:08.354161 4198 raft_consensus.cc:2454] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39fa89d23aef43ad87a152b487d0abda in term 1.
I20260811 03:00:08.354324 4608 leader_election.cc:302] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e; no voters:
I20260811 03:00:08.354480 4869 raft_consensus.cc:2781] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:08.354693 4869 raft_consensus.cc:686] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 1 LEADER]: Becoming Leader. State: Replica: 39fa89d23aef43ad87a152b487d0abda, State: Running, Role: LEADER
I20260811 03:00:08.354976 4869 consensus_queue.cc:237] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.358016 3901 catalog_manager.cc:5495] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda reported cstate change: term changed from 0 to 1, leader changed from <none> to 39fa89d23aef43ad87a152b487d0abda (127.8.203.67). New cstate: current_term: 1 leader_uuid: "39fa89d23aef43ad87a152b487d0abda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } health_report { overall_health: UNKNOWN } } }
I20260811 03:00:08.378654 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.375272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.8.203.67:43557)
0811 03:00:08.375386 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.375391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.375404 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:00:08.375570 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.375572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.375621 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.375706 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:00:08.375958 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.376435 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.377250 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.377503 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.377588 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.377887 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:00:08.378270 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:00:08.378372 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.378466 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.378515 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:00:08.384006 4198 raft_consensus.cc:1261] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Refusing update from remote peer 39fa89d23aef43ad87a152b487d0abda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:08.384006 4462 raft_consensus.cc:1261] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Refusing update from remote peer 39fa89d23aef43ad87a152b487d0abda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:08.384653 4869 consensus_queue.cc:1035] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [LEADER]: Connected to new peer: Peer: permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:08.384820 4875 consensus_queue.cc:1035] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:08.392413 4886 mvcc.cc:205] Tried to move back new op lower bound from 7317164885534203904 to 7317164885424283648. Current Snapshot: MvccSnapshot[applied={T|T < 7317164885534203904}]
I20260811 03:00:08.400825 3900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:08.400991 3900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:08.403343 3900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Sending DeleteTablet for 3 replicas of tablet 29edd748decb4594a1f4264560c973d0
I20260811 03:00:08.404160 4151 tablet_service.cc:1514] Processing DeleteTablet for tablet 29edd748decb4594a1f4264560c973d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:08 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:41834
I20260811 03:00:08.404153 4423 tablet_service.cc:1514] Processing DeleteTablet for tablet 29edd748decb4594a1f4264560c973d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:08 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:51822
I20260811 03:00:08.404222 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 4022
I20260811 03:00:08.417699 4902 tablet_replica.cc:330] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: stopping tablet replica
I20260811 03:00:08.418179 4902 raft_consensus.cc:2229] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:08.418591 4902 raft_consensus.cc:2258] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:08.418591 4688 tablet_service.cc:1514] Processing DeleteTablet for tablet 29edd748decb4594a1f4264560c973d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:08 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:35746
I20260811 03:00:08.419893 4905 tablet_replica.cc:330] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: stopping tablet replica
I20260811 03:00:08.420189 4905 raft_consensus.cc:2229] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:00:08.420564 4905 raft_consensus.cc:2258] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:08.422010 4902 ts_tablet_manager.cc:1905] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:08.422150 4905 ts_tablet_manager.cc:1905] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:08.424731 4902 ts_tablet_manager.cc:1918] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:08.424831 4902 log.cc:1196] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/29edd748decb4594a1f4264560c973d0
I20260811 03:00:08.424993 4905 ts_tablet_manager.cc:1918] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:08.425138 4905 log.cc:1196] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/29edd748decb4594a1f4264560c973d0
I20260811 03:00:08.425151 4902 ts_tablet_manager.cc:1939] T 29edd748decb4594a1f4264560c973d0 P b9bda3783ad8432c8369da6f0bd10a5e: Deleting consensus metadata
I20260811 03:00:08.425509 4905 ts_tablet_manager.cc:1939] T 29edd748decb4594a1f4264560c973d0 P 39fa89d23aef43ad87a152b487d0abda: Deleting consensus metadata
I20260811 03:00:08.426513 3858 catalog_manager.cc:4841] TS b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817): tablet 29edd748decb4594a1f4264560c973d0 (table test-table [id=d4ca294bf72e435483d68141dc1302bb]) successfully deleted
I20260811 03:00:08.426852 3856 catalog_manager.cc:4841] TS 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557): tablet 29edd748decb4594a1f4264560c973d0 (table test-table [id=d4ca294bf72e435483d68141dc1302bb]) successfully deleted
W20260811 03:00:08.427827 3862 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:34913 (error 108)
I20260811 03:00:08.428673 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 4291
I20260811 03:00:08.428900 4849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.428582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34913 (local address 127.0.0.1:41840)
0811 03:00:08.428727 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.428805 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:34913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 03:00:08.443917 3862 catalog_manager.cc:4584] TS 3313b669515a433387062842ed25d59a (127.8.203.65:34913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29edd748decb4594a1f4264560c973d0: Network error: Client connection negotiation failed: client connection to 127.8.203.65:34913: connect: Connection refused (error 111)
I20260811 03:00:08.449959 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 4575
I20260811 03:00:08.478910 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 3832
I20260811 03:00:08.499645 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:40397
--webserver_interface=127.8.203.126
--webserver_port=41049
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:08.620302 4911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:08.620559 4911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:08.620599 4911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:08.624558 4911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:08.624621 4911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:08.624639 4911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:08.624655 4911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:08.624667 4911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:08.629894 4911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=41049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:08.631021 4911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:08.632083 4911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:08.647132 4934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:08.647146 4939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:08.647367 4911 server_base.cc:1034] running on GCE node
W20260811 03:00:08.647507 4935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:08.648106 4911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:08.649261 4911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:08.650440 4911 hybrid_clock.cc:648] HybridClock initialized: now 1786417208650410 us; error 44 us; skew 500 ppm
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:08.653083 4911 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:00:08.654001 4911 webserver.cc:460] Webserver started at http://127.8.203.126:41049/ using document root <none> and password file <none>
I20260811 03:00:08.654554 4911 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:08.654624 4911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:08.658576 4911 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 03:00:08.660641 4950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.661890 4911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:08.661993 4911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "ac4b9b0b201340a98d84cb8e9bd304c2"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "e8c7c24cd714c5d5d28a251a5fb94d11"
server_key_iv: "266b6df4f80f792c01e76857783b82da"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.662570 4911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:08.680531 4911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:08.683156 4911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:08.683308 4911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:08.689900 4911 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:40397
I20260811 03:00:08.690155 5047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:40397 every 8 connection(s)
I20260811 03:00:08.690994 4911 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:00:08.695399 5048 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:00:08.697005 5048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Bootstrap starting.
I20260811 03:00:08.697327 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 4911
I20260811 03:00:08.697710 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:34913
--local_ip_for_outbound_sockets=127.8.203.65
--tserver_master_addrs=127.8.203.126:40397
--webserver_port=44673
--webserver_interface=127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:08.713696 5048 log.cc:824] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:08.722442 5048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:00:08.722762 5048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Bootstrap complete.
I20260811 03:00:08.726672 5048 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:08.727072 5048 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4b9b0b201340a98d84cb8e9bd304c2, State: Initialized, Role: FOLLOWER
I20260811 03:00:08.727553 5048 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:08.727679 5048 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:00:08.727749 5048 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:00:08.727852 5048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:00:08.729741 5048 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:08.730062 5048 leader_election.cc:302] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac4b9b0b201340a98d84cb8e9bd304c2; no voters:
I20260811 03:00:08.730357 5048 leader_election.cc:288] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 03:00:08.730464 5057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 03:00:08.730660 5057 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 LEADER]: Becoming Leader. State: Replica: ac4b9b0b201340a98d84cb8e9bd304c2, State: Running, Role: LEADER
I20260811 03:00:08.730953 5057 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:08.731622 5048 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:08.732288 5058 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } } }
I20260811 03:00:08.732440 5058 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:08.733060 5061 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac4b9b0b201340a98d84cb8e9bd304c2. Latest consensus state: current_term: 2 leader_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } } }
I20260811 03:00:08.733165 5061 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:08.733235 5070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:00:08.737959 5070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:00:08.738840 5070 catalog_manager.cc:1259] Loaded cluster ID: 6b11a1f226274db0adbdf251ba802854
I20260811 03:00:08.738900 5070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:00:08.740229 5070 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 03:00:08.744602 5070 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 03:00:08.749732 5070 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:00:08.750438 5070 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Loaded TSK: 0
I20260811 03:00:08.751621 5070 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:00:08.814404 5052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:08.814636 5052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:08.814677 5052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:08.818656 5052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:08.818711 5052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:08.818778 5052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:00:08.824216 5052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:34913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=44673
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:08.825228 5052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:08.826257 5052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:08.835569 5098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:08.835587 5095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:08.835577 5100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:08.835779 5052 server_base.cc:1034] running on GCE node
I20260811 03:00:08.836287 5052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:08.836907 5052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:08.838081 5052 hybrid_clock.cc:648] HybridClock initialized: now 1786417208838068 us; error 28 us; skew 500 ppm
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:08.840701 5052 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:00:08.841578 5052 webserver.cc:460] Webserver started at http://127.8.203.65:44673/ using document root <none> and password file <none>
I20260811 03:00:08.842105 5052 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:08.842175 5052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:08.846140 5052 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:00:08.848188 5112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:08.849399 5052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:00:08.849493 5052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "3313b669515a433387062842ed25d59a"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "f158c8107e5bc1da439f51edd09a74f0"
server_key_iv: "3f81cf1b5016818614bbb8659ae9872e"
server_key_version: "encryptionkey@0"
I20260811 03:00:08.850013 5052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:08.864701 5052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:08.867729 5052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:08.867887 5052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:08.868459 5052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:08.869475 5123 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 03:00:08.873386 5052 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 03:00:08.873446 5052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260811 03:00:08.873499 5052 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 03:00:08.875250 5052 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 03:00:08.875296 5052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 03:00:08.875550 5123 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Bootstrap starting.
I20260811 03:00:08.884974 5052 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:34913
I20260811 03:00:08.885062 5314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:34913 every 8 connection(s)
I20260811 03:00:08.885952 5052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:00:08.893601 5123 log.cc:824] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:08.895274 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 5052
I20260811 03:00:08.895650 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:43817
--local_ip_for_outbound_sockets=127.8.203.66
--tserver_master_addrs=127.8.203.126:40397
--webserver_port=33989
--webserver_interface=127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:08.901439 5123 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:00:08.901847 5123 tablet_bootstrap.cc:492] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Bootstrap complete.
I20260811 03:00:08.903007 5123 ts_tablet_manager.cc:1397] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Time spent bootstrapping tablet: real 0.028s user 0.022s sys 0.004s
I20260811 03:00:08.904138 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.887944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:55805 (local address 127.8.203.126:40397)
0811 03:00:08.888337 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.888363 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.888666 (+ 303us) server_negotiation.cc:407] Connection header received
0811 03:00:08.889932 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.889956 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.890411 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.890794 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:08.891950 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.893475 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.894869 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.895575 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.895738 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.898578 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:08.898584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:08.898596 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:08.898624 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:08.900502 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.901127 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.901134 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.901139 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.901316 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.901747 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.901751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.901752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.902221 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:08.902362 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.902960 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.903276 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":112,"threads_started":1}
I20260811 03:00:08.904136 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.887829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:55805)
0811 03:00:08.888490 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.888544 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.889394 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.890856 (+ 1462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.890863 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.891212 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:08.891764 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.891787 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.893628 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.893631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.894745 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.894756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.895949 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.895956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.896152 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.896816 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:08.896836 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:08.898393 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:08.900631 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.900637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.900651 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.900994 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.901417 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.901420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.901422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.901629 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.902350 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:08.902356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:08.902626 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.902813 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.903201 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":141,"threads_started":1}
I20260811 03:00:08.905859 5123 raft_consensus.cc:348] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.906358 5123 raft_consensus.cc:729] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3313b669515a433387062842ed25d59a, State: Initialized, Role: FOLLOWER
I20260811 03:00:08.906833 5123 consensus_queue.cc:260] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:08.907497 5319 heartbeater.cc:347] Connected to a master server at 127.8.203.126:40397
I20260811 03:00:08.907711 5319 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:08.907795 5123 ts_tablet_manager.cc:1428] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 03:00:08.908165 5319 heartbeater.cc:511] Master 127.8.203.126:40397 requested a full tablet report, sending...
I20260811 03:00:08.910681 4982 ts_manager.cc:194] Registered new tserver with Master: 3313b669515a433387062842ed25d59a (127.8.203.65:34913)
I20260811 03:00:08.914667 4982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:55805
I20260811 03:00:08.917061 5319 heartbeater.cc:503] Master 127.8.203.126:40397 was elected leader, sending a full tablet report...
Aug 11 03:00:08 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 03:00:08.924508 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.913951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34913 (local address 127.0.0.1:41848)
0811 03:00:08.914647 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:08.914682 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:08.914818 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:08.915256 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:08.915262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:08.915667 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:08.915793 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.915803 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.917243 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.917247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.918265 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:08.918278 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.919116 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.919119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:08.919205 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.919727 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:08.919739 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:08.921057 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:08.923010 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.923018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.923023 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.923240 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.923525 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:08.923527 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:08.923529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:08.923597 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:08.924035 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:08.924039 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:08.924132 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 03:00:08.924236 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 03:00:08.924380 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":114,"threads_started":1}
I20260811 03:00:08.924650 5328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:08.914001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.8.203.65:34913)
0811 03:00:08.914415 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:08.914427 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:08.914861 (+ 434us) server_negotiation.cc:407] Connection header received
0811 03:00:08.914950 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:08.914955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:08.915113 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:08.915233 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:08.915913 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.917102 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.918412 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:08.919005 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:08.919110 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:08.921190 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:08.921201 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:08.921215 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:08.921241 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:08.922911 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.923345 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.923350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.923352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.923445 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:08.923663 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:08.923666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:08.923667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:08.923943 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:08.924070 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:08.924337 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 03:00:08.924497 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":105,"threads_started":1}
I20260811 03:00:08.925598 5200 tablet_service.cc:1514] Processing DeleteTablet for tablet 29edd748decb4594a1f4264560c973d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:08 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:41848
I20260811 03:00:08.926409 5330 tablet_replica.cc:330] stopping tablet replica
I20260811 03:00:08.926688 5330 raft_consensus.cc:2229] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:08.926913 5330 raft_consensus.cc:2258] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:08.927835 5330 ts_tablet_manager.cc:1905] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:08.931268 5330 ts_tablet_manager.cc:1918] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:08.931348 5330 log.cc:1196] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/29edd748decb4594a1f4264560c973d0
I20260811 03:00:08.931595 5330 ts_tablet_manager.cc:1939] T 29edd748decb4594a1f4264560c973d0 P 3313b669515a433387062842ed25d59a: Deleting consensus metadata
I20260811 03:00:08.933171 4955 catalog_manager.cc:4841] TS 3313b669515a433387062842ed25d59a (127.8.203.65:34913): tablet 29edd748decb4594a1f4264560c973d0 (table test-table [id=d4ca294bf72e435483d68141dc1302bb]) successfully deleted
W20260811 03:00:09.014916 5324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:09.015161 5324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:09.015216 5324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:09.019162 5324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:09.019219 5324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:09.019291 5324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:00:09.024428 5324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:43817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=33989
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:09.025439 5324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:09.026415 5324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:09.036291 5350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:09.036305 5346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:09.036393 5324 server_base.cc:1034] running on GCE node
W20260811 03:00:09.036315 5347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:09.036916 5324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:09.037552 5324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:09.038707 5324 hybrid_clock.cc:648] HybridClock initialized: now 1786417209038690 us; error 27 us; skew 500 ppm
Aug 11 03:00:09 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:09.041234 5324 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:00:09.042066 5324 webserver.cc:460] Webserver started at http://127.8.203.66:33989/ using document root <none> and password file <none>
I20260811 03:00:09.042588 5324 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:09.042673 5324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:09.046365 5324 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 03:00:09.048308 5360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:09.049604 5324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260811 03:00:09.049718 5324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "b9bda3783ad8432c8369da6f0bd10a5e"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "7fa1ec02984b0edbcb82f84dd3d595cf"
server_key_iv: "7623275199282cab8c8d279b04ea8306"
server_key_version: "encryptionkey@0"
I20260811 03:00:09.050256 5324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:09.060721 5324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:09.063266 5324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:09.063431 5324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:09.063977 5324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:09.064764 5324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:09.064810 5324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:09.064860 5324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:09.064889 5324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:09.073213 5324 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:43817
I20260811 03:00:09.073223 5570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:43817 every 8 connection(s)
I20260811 03:00:09.074236 5324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 03:00:09 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:09.083724 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 5324
I20260811 03:00:09.084122 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:43557
--local_ip_for_outbound_sockets=127.8.203.67
--tserver_master_addrs=127.8.203.126:40397
--webserver_port=39857
--webserver_interface=127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:09.088117 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.075964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:41479 (local address 127.8.203.126:40397)
0811 03:00:09.076107 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.076111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.076946 (+ 835us) server_negotiation.cc:407] Connection header received
0811 03:00:09.078295 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.078299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.078357 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.078438 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:09.079645 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.080167 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.081276 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.081565 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.081656 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.083869 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:09.083873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:09.083876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:09.083902 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:09.085636 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:09.086241 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:09.086246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:09.086248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:09.086368 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:09.086782 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:09.086786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:09.086788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:09.087061 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:09.087163 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.087828 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.087961 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:00:09.089778 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.076363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:41479)
0811 03:00:09.076797 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:09.076840 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:09.078081 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:09.078623 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:09.078630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:09.079008 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:09.079495 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.079509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.080288 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.080291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.081157 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.081167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.081710 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.081714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.081884 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.082362 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:09.082377 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:09.083706 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:09.085761 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:09.085767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:09.085782 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:09.086125 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:09.086480 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:09.086483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:09.086485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:09.086678 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:09.087165 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:09.087199 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:09.087532 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 03:00:09.088382 (+ 850us) client_negotiation.cc:240] Negotiation successful
0811 03:00:09.088708 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
I20260811 03:00:09.094115 5571 heartbeater.cc:347] Connected to a master server at 127.8.203.126:40397
I20260811 03:00:09.094457 5571 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:09.095137 5571 heartbeater.cc:511] Master 127.8.203.126:40397 requested a full tablet report, sending...
I20260811 03:00:09.096470 4982 ts_manager.cc:194] Registered new tserver with Master: b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
I20260811 03:00:09.097379 4982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:41479
W20260811 03:00:09.198276 5575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:09.198518 5575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:09.198557 5575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:09.202472 5575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:09.202524 5575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:09.202579 5575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:00:09.207522 5575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:43557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=39857
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:09.208434 5575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:09.209375 5575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:09.218384 5587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:09.218443 5584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:09.218398 5585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:09.218566 5575 server_base.cc:1034] running on GCE node
I20260811 03:00:09.219033 5575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:09.219666 5575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:09.220808 5575 hybrid_clock.cc:648] HybridClock initialized: now 1786417209220788 us; error 40 us; skew 500 ppm
Aug 11 03:00:09 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:09.223315 5575 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:00:09.224129 5575 webserver.cc:460] Webserver started at http://127.8.203.67:39857/ using document root <none> and password file <none>
I20260811 03:00:09.224637 5575 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:09.224706 5575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:09.228350 5575 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:09.230321 5604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:09.231513 5575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:09.231650 5575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "39fa89d23aef43ad87a152b487d0abda"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "67e2d007ec72afbfb57b2df659ab77ee"
server_key_iv: "ba3a9714d4ec136dd8d5579877850692"
server_key_version: "encryptionkey@0"
I20260811 03:00:09.232157 5575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:09.264068 5575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:09.266963 5575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:09.267145 5575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:09.267755 5575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:09.268617 5575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:09.268676 5575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:09.268740 5575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:09.268770 5575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:09.277249 5575 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43557
I20260811 03:00:09.277379 5820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43557 every 8 connection(s)
I20260811 03:00:09.278273 5575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:00:09.282361 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 5575
Aug 11 03:00:09 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:09.290114 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.282984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.8.203.126:40397)
0811 03:00:09.283198 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.283208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.283377 (+ 169us) server_negotiation.cc:407] Connection header received
0811 03:00:09.283499 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.283502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.283544 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.283616 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:09.284302 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.284805 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.285670 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.285886 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.285968 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.287190 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:09.287194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:09.287197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:09.287223 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:09.288601 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:09.288983 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:09.288987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:09.288988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:09.289063 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:09.289319 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:09.289322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:09.289324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:09.289499 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:09.289700 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.289837 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.289946 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260811 03:00:09.291651 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.280014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:40899 (local address 127.8.203.126:40397)
0811 03:00:09.280168 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.280171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.280993 (+ 822us) server_negotiation.cc:407] Connection header received
0811 03:00:09.282355 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.282358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.282415 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.282486 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:09.283900 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.284409 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.285555 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.285773 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.285860 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.288380 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:09.288383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:09.288385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:09.288417 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:09.289686 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:09.290180 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:09.290184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:09.290185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:09.290269 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:09.290586 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:09.290590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:09.290591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:09.290786 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:09.290880 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.291394 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.291506 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 03:00:09.292497 5825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.280414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:40899)
0811 03:00:09.280844 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:09.280886 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:09.282144 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:09.282771 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:09.282780 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:09.283277 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:09.283760 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.283774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.284526 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.284529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.285444 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.285454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.285909 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.285913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.286074 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.286668 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:09.286684 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:09.288230 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:09.289793 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:09.289799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:09.289813 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:09.290063 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:09.290363 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:09.290366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:09.290367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:09.290496 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:09.290891 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:09.290897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:09.291133 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 03:00:09.291324 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 03:00:09.291627 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":98,"threads_started":1}
I20260811 03:00:09.293475 5822 heartbeater.cc:347] Connected to a master server at 127.8.203.126:40397
I20260811 03:00:09.293692 5822 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:09.294137 5822 heartbeater.cc:511] Master 127.8.203.126:40397 requested a full tablet report, sending...
I20260811 03:00:09.295071 4982 ts_manager.cc:194] Registered new tserver with Master: 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557)
I20260811 03:00:09.295603 4982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:40899
I20260811 03:00:09.303328 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:00:09.307415 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.303903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56556 (local address 127.8.203.126:40397)
0811 03:00:09.304024 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.304028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.304205 (+ 177us) server_negotiation.cc:407] Connection header received
0811 03:00:09.304360 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.304363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.304457 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.304544 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:00:09.304770 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.305258 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.306106 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.306311 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.306396 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.306685 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:00:09.307060 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:00:09.307164 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.307234 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.307283 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:00:09.308000 4982 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:00:09.311506 4982 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:00:09.318672 5200 tablet_service.cc:1467] Processing CreateTablet for tablet 38c3db3d86514f6cb7efd10a38796469 (DEFAULT_TABLE table=test-table [id=c51d10f2af96497b83a69d0b1e1969f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:09.319319 5200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c3db3d86514f6cb7efd10a38796469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:09.322229 5123 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Bootstrap starting.
I20260811 03:00:09.322906 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.317711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43557 (local address 127.0.0.1:35756)
0811 03:00:09.317805 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:09.317818 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:09.317886 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:09.319060 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:09.319063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:09.319082 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:09.319175 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.319181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.320588 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.320591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.321509 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.321521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.322502 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.322506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.322596 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.322646 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 03:00:09.322694 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 03:00:09.322768 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 03:00:09.323127 5123 tablet_bootstrap.cc:654] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:09.323598 5841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.317908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.8.203.67:43557)
0811 03:00:09.318589 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.318601 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.318628 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:00:09.318705 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.318709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.318902 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.319046 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:09.319434 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.320464 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.321652 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.322367 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.323166 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.323321 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.323373 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.323460 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":185,"threads_started":1}
I20260811 03:00:09.324810 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.318132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43817 (local address 127.0.0.1:51828)
0811 03:00:09.319271 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:09.319286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:09.319379 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:09.319921 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:09.319924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:09.319944 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:09.320038 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.320044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.321699 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.321702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.322815 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.322827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.324432 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.324436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.324522 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.324547 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 03:00:09.324610 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 03:00:09.324681 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260811 03:00:09.325409 5845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.318295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.8.203.66:43817)
0811 03:00:09.318611 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.318622 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.319365 (+ 743us) server_negotiation.cc:407] Connection header received
0811 03:00:09.319572 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.319577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.319769 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.319938 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:09.320133 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.321544 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.323155 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.324139 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.324224 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.324354 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.325142 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.325239 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":122,"threads_started":1}
I20260811 03:00:09.325824 5123 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: No bootstrap required, opened a new log
I20260811 03:00:09.325910 5123 ts_tablet_manager.cc:1397] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 03:00:09.326336 5123 raft_consensus.cc:348] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.326457 5123 raft_consensus.cc:374] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:09.326496 5123 raft_consensus.cc:729] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3313b669515a433387062842ed25d59a, State: Initialized, Role: FOLLOWER
I20260811 03:00:09.326615 5123 consensus_queue.cc:260] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.326874 5123 ts_tablet_manager.cc:1428] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 03:00:09.326792 5692 tablet_service.cc:1467] Processing CreateTablet for tablet 38c3db3d86514f6cb7efd10a38796469 (DEFAULT_TABLE table=test-table [id=c51d10f2af96497b83a69d0b1e1969f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:09.327950 5692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c3db3d86514f6cb7efd10a38796469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:09.328557 5455 tablet_service.cc:1467] Processing CreateTablet for tablet 38c3db3d86514f6cb7efd10a38796469 (DEFAULT_TABLE table=test-table [id=c51d10f2af96497b83a69d0b1e1969f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:09.329617 5455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38c3db3d86514f6cb7efd10a38796469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:09.336234 5850 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Bootstrap starting.
I20260811 03:00:09.337409 5852 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Bootstrap starting.
I20260811 03:00:09.339039 5850 tablet_bootstrap.cc:654] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:09.339638 5852 tablet_bootstrap.cc:654] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:09.339745 5850 log.cc:824] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:09.340389 5852 log.cc:824] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:09.341562 5850 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: No bootstrap required, opened a new log
I20260811 03:00:09.341735 5850 ts_tablet_manager.cc:1397] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 03:00:09.342125 5852 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: No bootstrap required, opened a new log
I20260811 03:00:09.342303 5852 ts_tablet_manager.cc:1397] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:00:09.344657 5850 raft_consensus.cc:348] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.344890 5850 raft_consensus.cc:374] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:09.344949 5850 raft_consensus.cc:729] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39fa89d23aef43ad87a152b487d0abda, State: Initialized, Role: FOLLOWER
I20260811 03:00:09.345036 5852 raft_consensus.cc:348] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.345253 5852 raft_consensus.cc:374] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:09.345323 5852 raft_consensus.cc:729] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9bda3783ad8432c8369da6f0bd10a5e, State: Initialized, Role: FOLLOWER
I20260811 03:00:09.345398 5850 consensus_queue.cc:260] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.345710 5852 consensus_queue.cc:260] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.346036 5822 heartbeater.cc:503] Master 127.8.203.126:40397 was elected leader, sending a full tablet report...
I20260811 03:00:09.346377 5850 ts_tablet_manager.cc:1428] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:00:09.346586 5852 ts_tablet_manager.cc:1428] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260811 03:00:09.346594 5571 heartbeater.cc:503] Master 127.8.203.126:40397 was elected leader, sending a full tablet report...
W20260811 03:00:09.387076 5320 tablet.cc:2369] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:00:09.529229 5824 tablet.cc:2369] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:00:09.575346 5573 tablet.cc:2369] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:00:09.656569 5325 raft_consensus.cc:482] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:09.656806 5325 raft_consensus.cc:504] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.657982 5325 leader_election.cc:288] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557), b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
I20260811 03:00:09.661813 5841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.658355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:53325 (local address 127.8.203.67:43557)
0811 03:00:09.658585 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.658589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.658606 (+ 17us) server_negotiation.cc:407] Connection header received
0811 03:00:09.658796 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.658799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.658844 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.658908 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:09.659214 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.659700 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.660792 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.661219 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.661359 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.661416 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.661614 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.661685 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 03:00:09.661813 5845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.658339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:36557 (local address 127.8.203.66:43817)
0811 03:00:09.658472 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.658476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.658688 (+ 212us) server_negotiation.cc:407] Connection header received
0811 03:00:09.658823 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.658825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.658868 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.658928 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:09.659213 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.659711 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.660793 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.661264 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.661397 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.661452 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.661602 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.661674 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:00:09.661824 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.658235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43557 (local address 127.8.203.65:53325)
0811 03:00:09.658537 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:09.658552 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:09.658655 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:09.658931 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:09.658934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:09.658951 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:09.659102 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.659107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.659806 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.659809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.660668 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.660677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.661359 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.661362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.661447 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.661518 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 03:00:09.661584 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 03:00:09.661685 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":70,"spinlock_wait_cycles":27520,"thread_start_us":89,"threads_started":1}
I20260811 03:00:09.661827 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.658235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43817 (local address 127.8.203.65:36557)
0811 03:00:09.658601 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:09.658614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:09.658722 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:09.658951 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:09.658953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:09.658978 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:09.659108 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.659114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.659817 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.659820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.660677 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:09.660687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.661392 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.661395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:09.661476 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.661514 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 03:00:09.661584 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 03:00:09.661674 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":121,"spinlock_wait_cycles":768,"thread_start_us":80,"threads_started":1}
I20260811 03:00:09.662422 5736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c3db3d86514f6cb7efd10a38796469" candidate_uuid: "3313b669515a433387062842ed25d59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39fa89d23aef43ad87a152b487d0abda" is_pre_election: true
I20260811 03:00:09.662596 5485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c3db3d86514f6cb7efd10a38796469" candidate_uuid: "3313b669515a433387062842ed25d59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" is_pre_election: true
I20260811 03:00:09.662765 5736 raft_consensus.cc:2454] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3313b669515a433387062842ed25d59a in term 0.
I20260811 03:00:09.662843 5485 raft_consensus.cc:2454] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3313b669515a433387062842ed25d59a in term 0.
I20260811 03:00:09.663203 5113 leader_election.cc:302] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3313b669515a433387062842ed25d59a, 39fa89d23aef43ad87a152b487d0abda; no voters:
I20260811 03:00:09.663486 5325 raft_consensus.cc:2781] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:00:09.663575 5325 raft_consensus.cc:482] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:00:09.663628 5325 raft_consensus.cc:3037] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:09.664623 5325 raft_consensus.cc:504] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.664933 5325 leader_election.cc:288] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 1 election: Requested vote from peers 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557), b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
I20260811 03:00:09.665331 5485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c3db3d86514f6cb7efd10a38796469" candidate_uuid: "3313b669515a433387062842ed25d59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9bda3783ad8432c8369da6f0bd10a5e"
I20260811 03:00:09.665412 5736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c3db3d86514f6cb7efd10a38796469" candidate_uuid: "3313b669515a433387062842ed25d59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39fa89d23aef43ad87a152b487d0abda"
I20260811 03:00:09.665452 5485 raft_consensus.cc:3037] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:09.665529 5736 raft_consensus.cc:3037] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:09.666553 5736 raft_consensus.cc:2454] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3313b669515a433387062842ed25d59a in term 1.
I20260811 03:00:09.666567 5485 raft_consensus.cc:2454] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3313b669515a433387062842ed25d59a in term 1.
I20260811 03:00:09.666817 5113 leader_election.cc:302] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3313b669515a433387062842ed25d59a, 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e; no voters:
I20260811 03:00:09.666982 5325 raft_consensus.cc:2781] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:09.667178 5325 raft_consensus.cc:686] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 LEADER]: Becoming Leader. State: Replica: 3313b669515a433387062842ed25d59a, State: Running, Role: LEADER
I20260811 03:00:09.667421 5325 consensus_queue.cc:237] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:09.670148 4982 catalog_manager.cc:5495] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3313b669515a433387062842ed25d59a (127.8.203.65). New cstate: current_term: 1 leader_uuid: "3313b669515a433387062842ed25d59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } health_report { overall_health: HEALTHY } } }
I20260811 03:00:09.691769 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.688183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48162 (local address 127.8.203.65:34913)
0811 03:00:09.688426 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:09.688433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:09.688444 (+ 11us) server_negotiation.cc:407] Connection header received
0811 03:00:09.688508 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:09.688510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:09.688558 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:09.688649 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:00:09.688844 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.689389 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.690257 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:09.690566 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:09.690674 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:09.690925 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:00:09.691404 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:00:09.691530 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:09.691571 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 03:00:09.691626 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260811 03:00:09.695785 5485 raft_consensus.cc:1261] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Refusing update from remote peer 3313b669515a433387062842ed25d59a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:09.695785 5736 raft_consensus.cc:1261] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 1 FOLLOWER]: Refusing update from remote peer 3313b669515a433387062842ed25d59a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:09.696542 5325 consensus_queue.cc:1035] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:09.696748 5860 consensus_queue.cc:1035] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:09.707760 5867 mvcc.cc:205] Tried to move back new op lower bound from 7317164890907299840 to 7317164890799927296. Current Snapshot: MvccSnapshot[applied={T|T < 7317164890907299840}]
I20260811 03:00:09.715708 4982 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:09.715886 4982 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:09.718202 4982 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Sending DeleteTablet for 3 replicas of tablet 38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:09.718873 5692 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c3db3d86514f6cb7efd10a38796469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:09 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:35756
I20260811 03:00:09.718890 5455 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c3db3d86514f6cb7efd10a38796469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:09 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:51828
I20260811 03:00:09.719019 5200 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c3db3d86514f6cb7efd10a38796469 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:09 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:41848
I20260811 03:00:09.719063 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 5052
I20260811 03:00:09.727826 5880 tablet_replica.cc:330] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: stopping tablet replica
I20260811 03:00:09.728245 5880 raft_consensus.cc:2229] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:09.728703 5880 raft_consensus.cc:2258] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:09.730261 5882 tablet_replica.cc:330] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: stopping tablet replica
I20260811 03:00:09.730577 5882 raft_consensus.cc:2229] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:09.730631 5880 ts_tablet_manager.cc:1905] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:09.730870 5882 raft_consensus.cc:2258] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:09.732275 5882 ts_tablet_manager.cc:1905] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:09.733716 5880 ts_tablet_manager.cc:1918] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:09.733822 5880 log.cc:1196] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:09.734171 5880 ts_tablet_manager.cc:1939] T 38c3db3d86514f6cb7efd10a38796469 P b9bda3783ad8432c8369da6f0bd10a5e: Deleting consensus metadata
I20260811 03:00:09.734849 5882 ts_tablet_manager.cc:1918] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:09.734958 5882 log.cc:1196] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:09.735260 5882 ts_tablet_manager.cc:1939] T 38c3db3d86514f6cb7efd10a38796469 P 39fa89d23aef43ad87a152b487d0abda: Deleting consensus metadata
I20260811 03:00:09.735517 4953 catalog_manager.cc:4841] TS b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817): tablet 38c3db3d86514f6cb7efd10a38796469 (table test-table [id=c51d10f2af96497b83a69d0b1e1969f8]) successfully deleted
I20260811 03:00:09.736336 4952 catalog_manager.cc:4841] TS 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557): tablet 38c3db3d86514f6cb7efd10a38796469 (table test-table [id=c51d10f2af96497b83a69d0b1e1969f8]) successfully deleted
W20260811 03:00:09.737136 4955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:34913 (error 108)
I20260811 03:00:09.737959 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 5324
I20260811 03:00:09.738159 5843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:09.737866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34913 (local address 127.0.0.1:48170)
0811 03:00:09.738001 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:09.738073 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:34913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 03:00:09.745375 4955 catalog_manager.cc:4584] TS 3313b669515a433387062842ed25d59a (127.8.203.65:34913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38c3db3d86514f6cb7efd10a38796469: Network error: Client connection negotiation failed: client connection to 127.8.203.65:34913: connect: Connection refused (error 111)
I20260811 03:00:09.753225 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 5575
I20260811 03:00:09.767652 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 4911
I20260811 03:00:09.782660 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:40397
--webserver_interface=127.8.203.126
--webserver_port=41049
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:09.896103 5884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:09.896366 5884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:09.896423 5884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:09.900336 5884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:09.900389 5884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:09.900415 5884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:09.900436 5884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:09.900462 5884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:09.905596 5884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=41049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:09.906569 5884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:09.907547 5884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:09.915799 5892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:09.915854 5889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:09.916000 5884 server_base.cc:1034] running on GCE node
W20260811 03:00:09.915864 5890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:09.916666 5884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:09.917765 5884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:09.918918 5884 hybrid_clock.cc:648] HybridClock initialized: now 1786417209918886 us; error 44 us; skew 500 ppm
Aug 11 03:00:09 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:09.921486 5884 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:00:09.922322 5884 webserver.cc:460] Webserver started at http://127.8.203.126:41049/ using document root <none> and password file <none>
I20260811 03:00:09.922892 5884 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:09.922959 5884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:09.926606 5884 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:00:09.928578 5904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:09.929780 5884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:00:09.929904 5884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "ac4b9b0b201340a98d84cb8e9bd304c2"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "e8c7c24cd714c5d5d28a251a5fb94d11"
server_key_iv: "266b6df4f80f792c01e76857783b82da"
server_key_version: "encryptionkey@0"
I20260811 03:00:09.930409 5884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:09.953967 5884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:09.956686 5884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:09.956867 5884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:09.963613 5884 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:40397
I20260811 03:00:09.963860 6001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:40397 every 8 connection(s)
I20260811 03:00:09.964658 5884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:00:09.968835 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 5884
I20260811 03:00:09.968954 6002 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:00:09.969308 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:34913
--local_ip_for_outbound_sockets=127.8.203.65
--tserver_master_addrs=127.8.203.126:40397
--webserver_port=44673
--webserver_interface=127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:09.971006 6002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Bootstrap starting.
I20260811 03:00:09.988054 6002 log.cc:824] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:09.999554 6002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:00:09.999904 6002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2: Bootstrap complete.
I20260811 03:00:10.003829 6002 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:10.004262 6002 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac4b9b0b201340a98d84cb8e9bd304c2, State: Initialized, Role: FOLLOWER
I20260811 03:00:10.004755 6002 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:10.004881 6002 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:00:10.004941 6002 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:00:10.005038 6002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 2 FOLLOWER]: Advancing to term 3
I20260811 03:00:10.006946 6002 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:10.007262 6002 leader_election.cc:302] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac4b9b0b201340a98d84cb8e9bd304c2; no voters:
I20260811 03:00:10.007570 6002 leader_election.cc:288] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 03:00:10.007670 6011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 03:00:10.007863 6011 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [term 3 LEADER]: Becoming Leader. State: Replica: ac4b9b0b201340a98d84cb8e9bd304c2, State: Running, Role: LEADER
I20260811 03:00:10.008162 6011 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } }
I20260811 03:00:10.008827 6002 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:10.009097 6011 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac4b9b0b201340a98d84cb8e9bd304c2. Latest consensus state: current_term: 3 leader_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } } }
I20260811 03:00:10.009208 6011 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:10.009601 6013 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac4b9b0b201340a98d84cb8e9bd304c2" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 40397 } } }
I20260811 03:00:10.009759 6013 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac4b9b0b201340a98d84cb8e9bd304c2 [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:10.010859 6019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:00:10.016146 6019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:00:10.016752 6019 catalog_manager.cc:1259] Loaded cluster ID: 6b11a1f226274db0adbdf251ba802854
I20260811 03:00:10.016808 6019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 03:00:10.017885 6019 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6d3ce379ad google::LogMessage::Fail() at ??:0
@ 0x7f6d3ce3a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6d3ce374de google::LogMessage::Flush() at ??:0
@ 0x7f6d3ce38a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6d47776fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6d47777d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6d47774579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6d477b8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6d3f0893d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6d3f089c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6d3f08b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6d3f07a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6d406be6db start_thread at ??:0
@ 0x7f6d3b93561f clone at ??:0
W20260811 03:00:10.106870 6008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:10.107108 6008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:10.107151 6008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:10.111078 6008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:10.111128 6008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:10.111186 6008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:00:10.116341 6008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:34913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=44673
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:10.117398 6008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:10.118386 6008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:10.127895 6081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:10.127920 6082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:10.128217 6008 server_base.cc:1034] running on GCE node
W20260811 03:00:10.127915 6084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:10.128701 6008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:10.129346 6008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:10.130522 6008 hybrid_clock.cc:648] HybridClock initialized: now 1786417210130486 us; error 52 us; skew 500 ppm
Aug 11 03:00:10 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417210, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:10.133312 6008 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:00:10.134148 6008 webserver.cc:460] Webserver started at http://127.8.203.65:44673/ using document root <none> and password file <none>
I20260811 03:00:10.134686 6008 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:10.134757 6008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:10.138613 6008 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:00:10.140762 6098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:10.142114 6008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:10.142230 6008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "3313b669515a433387062842ed25d59a"
format_stamp: "Formatted at 2026-08-11 03:00:07 on dist-test-slave-gjw7"
server_key: "f158c8107e5bc1da439f51edd09a74f0"
server_key_iv: "3f81cf1b5016818614bbb8659ae9872e"
server_key_version: "encryptionkey@0"
I20260811 03:00:10.142752 6008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:10.165894 6008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:10.169029 6008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:10.169210 6008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:10.169816 6008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:10.170903 6109 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 03:00:10.176385 6008 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 03:00:10.176450 6008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 03:00:10.176520 6008 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 03:00:10.178207 6008 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 03:00:10.178253 6008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 03:00:10.178584 6109 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Bootstrap starting.
I20260811 03:00:10.188270 6008 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:34913
I20260811 03:00:10.188318 6300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:34913 every 8 connection(s)
I20260811 03:00:10.189441 6008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:00:10.197394 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 6008
I20260811 03:00:10.197871 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:43817
--local_ip_for_outbound_sockets=127.8.203.66
--tserver_master_addrs=127.8.203.126:40397
--webserver_port=33989
--webserver_interface=127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:10.198165 6109 log.cc:824] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:10.215937 6109 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:00:10.216398 6109 tablet_bootstrap.cc:492] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Bootstrap complete.
I20260811 03:00:10.217666 6109 ts_tablet_manager.cc:1397] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Time spent bootstrapping tablet: real 0.039s user 0.016s sys 0.013s
I20260811 03:00:10.220984 6109 raft_consensus.cc:348] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:10.221500 6109 raft_consensus.cc:729] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3313b669515a433387062842ed25d59a, State: Initialized, Role: FOLLOWER
I20260811 03:00:10.221994 6109 consensus_queue.cc:260] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
W20260811 03:00:10.222056 6304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:00:10.191809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:53295)
0811 03:00:10.192486 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.192547 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:10.193644 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:10.221018 (+ 27374us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":163,"threads_started":1}
W20260811 03:00:10.222671 6301 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:40397 (0 consecutive failures): Network error: Failed to ping master at 127.8.203.126:40397: Client connection negotiation failed: client connection to 127.8.203.126:40397: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 03:00:10.223119 6109 ts_tablet_manager.cc:1428] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 03:00:10.223483 6304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.223158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:53041)
0811 03:00:10.223345 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.223414 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 03:00:10.224186 6304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.223968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:47121)
0811 03:00:10.224082 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.224130 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 03:00:10.224349 6301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:00:10.317655 6307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:10.317924 6307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:10.317967 6307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:10.321902 6307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:10.321961 6307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:10.322019 6307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:00:10.327409 6307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:43817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=33989
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:10.328366 6307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:10.329365 6307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:10.338774 6316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:10.338799 6320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:10.338837 6317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:10.339124 6307 server_base.cc:1034] running on GCE node
I20260811 03:00:10.339501 6307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:10.340090 6307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:10.341261 6307 hybrid_clock.cc:648] HybridClock initialized: now 1786417210341239 us; error 31 us; skew 500 ppm
Aug 11 03:00:10 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417210, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:10.343971 6307 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:00:10.344847 6307 webserver.cc:460] Webserver started at http://127.8.203.66:33989/ using document root <none> and password file <none>
I20260811 03:00:10.345445 6307 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:10.345525 6307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:10.349354 6307 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260811 03:00:10.351348 6329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:10.352511 6307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 03:00:10.352640 6307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "b9bda3783ad8432c8369da6f0bd10a5e"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "7fa1ec02984b0edbcb82f84dd3d595cf"
server_key_iv: "7623275199282cab8c8d279b04ea8306"
server_key_version: "encryptionkey@0"
I20260811 03:00:10.353190 6307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:10.378086 6307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:10.381561 6307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:10.381784 6307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:10.382436 6307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:10.383347 6307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:10.383402 6307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:10.383466 6307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:10.383498 6307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:10.393019 6307 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:43817
I20260811 03:00:10.393041 6511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:43817 every 8 connection(s)
I20260811 03:00:10.394088 6307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:00:10.398162 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.396257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:58317)
0811 03:00:10.396738 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.396852 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
I20260811 03:00:10.398567 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 6307
W20260811 03:00:10.398660 6514 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:40397 (0 consecutive failures): Network error: Failed to ping master at 127.8.203.126:40397: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
I20260811 03:00:10.398988 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:43557
--local_ip_for_outbound_sockets=127.8.203.67
--tserver_master_addrs=127.8.203.126:40397
--webserver_port=39857
--webserver_interface=127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:10.399360 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.399049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:54445)
0811 03:00:10.399241 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.399297 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 03:00:10.399972 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.399765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:38873)
0811 03:00:10.399864 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.399911 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 03:00:10.400152 6514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 03:00:10.513038 6517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:10.513336 6517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:10.513408 6517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:10.517483 6517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:10.517544 6517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:10.517638 6517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:00:10.522832 6517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:43557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=39857
--tserver_master_addrs=127.8.203.126:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:10.523901 6517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:10.524947 6517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:10.534427 6538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:10.534425 6530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:10.534425 6541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:10.534518 6517 server_base.cc:1034] running on GCE node
I20260811 03:00:10.535079 6517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:10.535696 6517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:10.536852 6517 hybrid_clock.cc:648] HybridClock initialized: now 1786417210536823 us; error 44 us; skew 500 ppm
Aug 11 03:00:10 dist-test-slave-gjw7 krb5kdc[3811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417210, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:10.539470 6517 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:00:10.540284 6517 webserver.cc:460] Webserver started at http://127.8.203.67:39857/ using document root <none> and password file <none>
I20260811 03:00:10.540819 6517 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:10.540890 6517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:10.544723 6517 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:10.547451 6552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:10.548624 6517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:00:10.548722 6517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "39fa89d23aef43ad87a152b487d0abda"
format_stamp: "Formatted at 2026-08-11 03:00:08 on dist-test-slave-gjw7"
server_key: "67e2d007ec72afbfb57b2df659ab77ee"
server_key_iv: "ba3a9714d4ec136dd8d5579877850692"
server_key_version: "encryptionkey@0"
I20260811 03:00:10.549322 6517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:10.565102 6517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:10.567915 6517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:10.568091 6517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:10.568717 6517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:10.569559 6517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:10.569608 6517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:10.569662 6517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:10.569687 6517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:10.577764 6517 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43557
I20260811 03:00:10.577821 6747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43557 every 8 connection(s)
I20260811 03:00:10.578737 6517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:00:10.582695 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.580862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:40173)
0811 03:00:10.581398 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.581522 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":115,"threads_started":1}
W20260811 03:00:10.583129 6748 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:40397 (0 consecutive failures): Network error: Failed to ping master at 127.8.203.126:40397: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
I20260811 03:00:10.583748 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.583516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:48899)
0811 03:00:10.583636 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.583692 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 03:00:10.584334 6751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:10.584121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:51989)
0811 03:00:10.584232 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:10.584280 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 03:00:10.584503 6748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 03:00:10.586700 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 6517
I20260811 03:00:11.225379 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:11.224911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:41483)
0811 03:00:11.225186 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:11.225242 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
I20260811 03:00:11.401015 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:11.400632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:53797)
0811 03:00:11.400867 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:11.400916 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":79,"threads_started":1}
I20260811 03:00:11.585363 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:11.584976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:56917)
0811 03:00:11.585187 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:11.585236 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
I20260811 03:00:11.921777 6757 raft_consensus.cc:482] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:11.921967 6757 raft_consensus.cc:504] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:11.922973 6757 leader_election.cc:288] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557), b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
Aug 11 03:00:11 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417210, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.66@KRBTEST.COM
Aug 11 03:00:11 dist-test-slave-gjw7 krb5kdc[3811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417210, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.67@KRBTEST.COM
I20260811 03:00:11.939005 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:11.923194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43817 (local address 127.8.203.65:48419)
0811 03:00:11.923513 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:11.923527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:11.923635 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:11.925170 (+ 1535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:11.925176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:11.925772 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:11.926255 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:11.926272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.927590 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.927593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:11.928511 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:11.928520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.929192 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.929196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:11.929430 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:11.932877 (+ 3447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:11.932893 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:11.934474 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:11.936604 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:11.936610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:11.936623 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:11.936983 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:11.937342 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:11.937346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:11.937348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:11.937475 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:11.938059 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:11.938065 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:11.938302 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 03:00:11.938454 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 03:00:11.938789 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":200,"spinlock_wait_cycles":43264,"thread_start_us":59,"threads_started":1}
I20260811 03:00:11.939018 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:11.923400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:48419 (local address 127.8.203.66:43817)
0811 03:00:11.923769 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:11.923781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:11.923817 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:00:11.924483 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:11.924489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:11.924823 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:11.925097 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:11.926429 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.927455 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.928627 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.929075 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.929229 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:11.934702 (+ 5473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:11.934709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:11.934725 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:11.934760 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:11.936499 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:11.937105 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:11.937111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:11.937117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:11.937227 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:11.937563 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:11.937566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:11.937568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:11.937948 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:11.938066 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:11.938602 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 03:00:11.938853 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":60,"threads_started":1}
I20260811 03:00:11.939036 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:11.923145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43557 (local address 127.8.203.65:52997)
0811 03:00:11.923452 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:11.923465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:11.923570 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:11.925170 (+ 1600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:11.925176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:11.925805 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:11.926255 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:11.926271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.927528 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.927532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:11.928513 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:11.928524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.929190 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.929194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:11.929429 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:11.932892 (+ 3463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:11.932905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:11.934823 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:11.936683 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:11.936686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:11.936687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:11.936965 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:11.937318 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:11.937322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:11.937324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:11.937468 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:11.938059 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:11.938065 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:11.938302 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 03:00:11.938453 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 03:00:11.938870 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":292,"spinlock_wait_cycles":7552,"thread_start_us":83,"threads_started":1}
I20260811 03:00:11.939042 6761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:11.923358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:52997 (local address 127.8.203.67:43557)
0811 03:00:11.923775 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:11.923796 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:11.923830 (+ 34us) server_negotiation.cc:407] Connection header received
0811 03:00:11.924493 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:11.924498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:11.924823 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:11.925097 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:11.926430 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.927391 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.928628 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:11.929075 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:11.929221 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:11.935007 (+ 5786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:11.935011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:11.935022 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:11.935049 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:11.936576 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:11.937074 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:11.937079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:11.937084 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:11.937201 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:11.937560 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:11.937563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:11.937564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:11.937946 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:11.938066 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:11.938606 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 03:00:11.938858 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":106,"threads_started":1}
W20260811 03:00:11.941581 6100 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
W20260811 03:00:11.942338 6099 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:11.942454 6099 leader_election.cc:302] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3313b669515a433387062842ed25d59a; no voters: 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e
I20260811 03:00:11.942708 6757 raft_consensus.cc:2726] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:00:12.226439 6758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:12.226090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:50789)
0811 03:00:12.226254 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:12.226327 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:00:12.402041 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:12.401653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:52717)
0811 03:00:12.401892 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:12.401943 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":80,"threads_started":1}
I20260811 03:00:12.586352 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:12.585963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:52641)
0811 03:00:12.586205 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:12.586257 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260811 03:00:13.227568 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:13.227151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:34635)
0811 03:00:13.227410 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:13.227470 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":78,"threads_started":1}
I20260811 03:00:13.403205 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:13.402767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:41469)
0811 03:00:13.403045 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:13.403106 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260811 03:00:13.587352 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:13.586972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:40669)
0811 03:00:13.587207 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:13.587258 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":69,"threads_started":1}
I20260811 03:00:13.888803 6771 raft_consensus.cc:482] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:13.888927 6771 raft_consensus.cc:504] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:13.889413 6771 leader_election.cc:288] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557), b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
W20260811 03:00:13.890328 6100 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
W20260811 03:00:13.890444 6099 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:13.890551 6099 leader_election.cc:302] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3313b669515a433387062842ed25d59a; no voters: 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e
I20260811 03:00:13.890817 6771 raft_consensus.cc:2726] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:00:14.228694 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:14.228280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:45219)
0811 03:00:14.228544 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:14.228596 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":105,"threads_started":1}
I20260811 03:00:14.404223 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:14.403863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:34119)
0811 03:00:14.404088 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:14.404136 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
I20260811 03:00:14.588351 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:14.587976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:44253)
0811 03:00:14.588210 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:14.588263 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":71,"threads_started":1}
I20260811 03:00:15.229975 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:15.229534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:35451)
0811 03:00:15.229824 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:15.229879 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":72,"threads_started":1}
I20260811 03:00:15.405282 6783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:15.404910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:57109)
0811 03:00:15.405144 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:15.405196 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260811 03:00:15.589423 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:15.589005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:57873)
0811 03:00:15.589249 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:15.589330 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
I20260811 03:00:16.231040 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:16.230635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:33591)
0811 03:00:16.230891 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:16.230943 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":89,"threads_started":1}
I20260811 03:00:16.406329 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:16.405948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:35063)
0811 03:00:16.406190 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:16.406241 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":87,"threads_started":1}
I20260811 03:00:16.482283 6790 raft_consensus.cc:482] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:16.482414 6790 raft_consensus.cc:504] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:16.482806 6790 leader_election.cc:288] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557), b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
W20260811 03:00:16.483558 6099 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
W20260811 03:00:16.483678 6100 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:16.483737 6100 leader_election.cc:302] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3313b669515a433387062842ed25d59a; no voters: 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e
I20260811 03:00:16.483878 6790 raft_consensus.cc:2726] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:00:16.590466 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:16.590049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:51681)
0811 03:00:16.590309 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:16.590363 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260811 03:00:17.232111 6794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:17.231711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:41943)
0811 03:00:17.231958 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:17.232014 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260811 03:00:17.407385 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:17.406953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:37351)
0811 03:00:17.407248 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:17.407298 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":86,"threads_started":1}
I20260811 03:00:17.591568 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:17.591113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:53495)
0811 03:00:17.591406 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:17.591460 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":70,"threads_started":1}
I20260811 03:00:18.233206 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:18.232790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:48207)
0811 03:00:18.233052 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:18.233106 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
I20260811 03:00:18.408447 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:18.408019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:40849)
0811 03:00:18.408299 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:18.408356 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
I20260811 03:00:18.592592 6806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:18.592193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:40675)
0811 03:00:18.592435 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:18.592485 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260811 03:00:18.915903 6808 raft_consensus.cc:482] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:18.916047 6808 raft_consensus.cc:504] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:18.916453 6808 leader_election.cc:288] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557), b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
W20260811 03:00:18.917191 6100 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
W20260811 03:00:18.917320 6099 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:18.917407 6099 leader_election.cc:302] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3313b669515a433387062842ed25d59a; no voters: 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e
I20260811 03:00:18.917546 6808 raft_consensus.cc:2726] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:00:19.234356 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:19.233906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:47971)
0811 03:00:19.234190 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:19.234244 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260811 03:00:19.409538 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:19.409134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:33907)
0811 03:00:19.409397 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:19.409448 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260811 03:00:19.593588 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:19.593189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:51211)
0811 03:00:19.593445 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:19.593498 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":89,"threads_started":1}
I20260811 03:00:20.235627 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:20.235168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:36315)
0811 03:00:20.235442 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:20.235508 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260811 03:00:20.410547 6819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:20.410165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:51245)
0811 03:00:20.410408 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:20.410458 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
I20260811 03:00:20.594590 6821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:20.594227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:39831)
0811 03:00:20.594448 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:20.594500 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":69,"threads_started":1}
I20260811 03:00:21.236801 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:21.236329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:35243)
0811 03:00:21.236627 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:21.236697 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":93,"threads_started":1}
I20260811 03:00:21.411558 6826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:21.411184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:36437)
0811 03:00:21.411416 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:21.411466 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":86,"threads_started":1}
I20260811 03:00:21.595778 6831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:21.595292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:34685)
0811 03:00:21.595588 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:21.595653 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260811 03:00:22.237895 6832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:22.237490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:34589)
0811 03:00:22.237738 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:22.237791 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":73,"threads_started":1}
I20260811 03:00:22.412560 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:22.412176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:36511)
0811 03:00:22.412414 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:22.412465 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":69,"threads_started":1}
I20260811 03:00:22.596941 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:22.596518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:38815)
0811 03:00:22.596788 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:22.596845 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
I20260811 03:00:23.238982 6848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:23.238578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:58043)
0811 03:00:23.238821 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:23.238884 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":78,"threads_started":1}
I20260811 03:00:23.413582 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:23.413175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:46991)
0811 03:00:23.413441 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:23.413492 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":107,"threads_started":1}
I20260811 03:00:23.598095 6850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:23.597686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:41689)
0811 03:00:23.597939 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:23.597995 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
I20260811 03:00:24.240054 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:24.239640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:52253)
0811 03:00:24.239899 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:24.239955 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":76,"threads_started":1}
I20260811 03:00:24.414605 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:24.414218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:58625)
0811 03:00:24.414463 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:24.414514 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260811 03:00:24.583523 6857 raft_consensus.cc:482] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:24.583650 6857 raft_consensus.cc:504] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39fa89d23aef43ad87a152b487d0abda" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43557 } } peers { permanent_uuid: "b9bda3783ad8432c8369da6f0bd10a5e" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43817 } } peers { permanent_uuid: "3313b669515a433387062842ed25d59a" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34913 } }
I20260811 03:00:24.584044 6857 leader_election.cc:288] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557), b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817)
W20260811 03:00:24.584731 6099 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39fa89d23aef43ad87a152b487d0abda (127.8.203.67:43557): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
W20260811 03:00:24.584836 6100 leader_election.cc:341] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9bda3783ad8432c8369da6f0bd10a5e (127.8.203.66:43817): Not found: Tablet not found: 38c3db3d86514f6cb7efd10a38796469
I20260811 03:00:24.584923 6100 leader_election.cc:302] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3313b669515a433387062842ed25d59a; no voters: 39fa89d23aef43ad87a152b487d0abda, b9bda3783ad8432c8369da6f0bd10a5e
I20260811 03:00:24.585074 6857 raft_consensus.cc:2726] T 38c3db3d86514f6cb7efd10a38796469 P 3313b669515a433387062842ed25d59a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:00:24.599126 6860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:24.598737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:39197)
0811 03:00:24.598986 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:24.599037 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":91,"threads_started":1}
I20260811 03:00:25.241281 6862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:25.240905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.65:52669)
0811 03:00:25.241134 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:25.241186 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260811 03:00:25.415750 6865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:25.415365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.66:38213)
0811 03:00:25.415608 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:25.415660 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260811 03:00:25.596428 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 6008
I20260811 03:00:25.609055 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:25.607988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:40397 (local address 127.8.203.67:38845)
0811 03:00:25.608878 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:25.608943 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":714,"threads_started":1}
I20260811 03:00:25.626245 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 6307
I20260811 03:00:25.636385 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 6517
2026-08-11T03:00:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19610 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:00:25 dist-test-slave-gjw7 krb5kdc[6874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:00:25 dist-test-slave-gjw7 krb5kdc[6874](info): set up 2 sockets
Aug 11 03:00:25 dist-test-slave-gjw7 krb5kdc[6874](info): commencing operation
krb5kdc: starting...
W20260811 03:00:26.849264 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:00:26 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:00:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:00:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:00:26.964233 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41919
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:41919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:27.076134 6904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:27.076395 6904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:27.076434 6904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:27.080420 6904 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 03:00:27.080472 6904 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 03:00:27.080489 6904 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 03:00:27.080509 6904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:27.080535 6904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:27.080556 6904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:27.080579 6904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:27.080601 6904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:27.085714 6904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:41919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:27.086642 6904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:27.087594 6904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:27.095755 6921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.095772 6918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.095831 6917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:27.096050 6904 server_base.cc:1034] running on GCE node
I20260811 03:00:27.096597 6904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:27.097651 6904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:27.098802 6904 hybrid_clock.cc:648] HybridClock initialized: now 1786417227098788 us; error 26 us; skew 500 ppm
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417227, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:27.101454 6904 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:00:27.102329 6904 webserver.cc:460] Webserver started at http://127.8.203.126:38175/ using document root <none> and password file <none>
I20260811 03:00:27.102880 6904 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:27.102944 6904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:27.103140 6904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:27.105086 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "d1829f20ad1e467daf847875f99a238a"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "d5308f5aaa5ec21226af834f297826a3"
server_key_iv: "cbfeb25ec2bb2d3ac61cae667aea17b5"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.105661 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "d1829f20ad1e467daf847875f99a238a"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "d5308f5aaa5ec21226af834f297826a3"
server_key_iv: "cbfeb25ec2bb2d3ac61cae667aea17b5"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.109102 6904 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:00:27.111500 6933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.112649 6904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:27.112761 6904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "d1829f20ad1e467daf847875f99a238a"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "d5308f5aaa5ec21226af834f297826a3"
server_key_iv: "cbfeb25ec2bb2d3ac61cae667aea17b5"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.112850 6904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:27.133991 6904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:27.136893 6904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:27.137060 6904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:27.143673 6904 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:41919
I20260811 03:00:27.143920 7023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:41919 every 8 connection(s)
I20260811 03:00:27.144573 6904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:00:27.147176 7025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:27.150135 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 6904
I20260811 03:00:27.150242 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:00:27.150484 9005 external_mini_cluster.cc:1428] Setting key ff1aa5708074e8380c85a96503520c89
I20260811 03:00:27.152710 7025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Bootstrap starting.
I20260811 03:00:27.155118 7025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:27.155911 7025 log.cc:824] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:27.157950 7025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: No bootstrap required, opened a new log
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:27.161680 7025 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:27.161936 7025 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:27.162011 7025 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1829f20ad1e467daf847875f99a238a, State: Initialized, Role: FOLLOWER
I20260811 03:00:27.162451 7025 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:27.162593 7025 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:00:27.162665 7025 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:00:27.162781 7025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:27.163813 7025 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:27.164170 7025 leader_election.cc:302] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1829f20ad1e467daf847875f99a238a; no voters:
I20260811 03:00:27.164500 7025 leader_election.cc:288] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:00:27.164625 7036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:27.164640 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.151767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.8.203.126:41919)
0811 03:00:27.152208 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.152222 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.152257 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:00:27.152894 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.152912 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.153226 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.153560 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:27.154431 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.155437 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.156551 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.157027 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.157144 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.159781 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:27.159787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:27.159799 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:27.159825 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:27.161351 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.161775 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.161780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.161784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.161891 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.162178 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.162181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.162182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.162569 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:27.162775 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.163353 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.163676 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":122,"threads_started":1}
I20260811 03:00:27.164849 7036 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 LEADER]: Becoming Leader. State: Replica: d1829f20ad1e467daf847875f99a238a, State: Running, Role: LEADER
I20260811 03:00:27.165160 7036 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:27.165620 7025 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:27.166378 7038 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1829f20ad1e467daf847875f99a238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } } }
I20260811 03:00:27.166388 7039 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1829f20ad1e467daf847875f99a238a. Latest consensus state: current_term: 1 leader_uuid: "d1829f20ad1e467daf847875f99a238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } } }
I20260811 03:00:27.166661 7039 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:27.166893 7038 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:27.167104 7052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:00:27.170282 7052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:00:27.175524 7052 catalog_manager.cc:1347] Generated new cluster ID: a4d008e7c9a94f238c185328caa038af
I20260811 03:00:27.175585 7052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:00:27.186298 7052 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:00:27.187140 7052 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:00:27.191309 7052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Generated new TSK 0
I20260811 03:00:27.191891 7052 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 03:00:27.219763 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41919
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:27.331935 7069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:27.332192 7069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:27.332235 7069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:27.336162 7069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:27.336220 7069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:27.336277 7069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:00:27.341352 7069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:27.342314 7069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:27.343279 7069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:27.352464 7080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.352474 7077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.352474 7087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:27.352653 7069 server_base.cc:1034] running on GCE node
I20260811 03:00:27.353066 7069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:27.353765 7069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:27.354924 7069 hybrid_clock.cc:648] HybridClock initialized: now 1786417227354905 us; error 29 us; skew 500 ppm
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417227, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:27.357488 7069 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:00:27.358311 7069 webserver.cc:460] Webserver started at http://127.8.203.65:33109/ using document root <none> and password file <none>
I20260811 03:00:27.358846 7069 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:27.358911 7069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:27.359088 7069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:27.360996 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "34b51efa70c449429918a62cc6d934c9"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "ecfae09fe54afe55f234f0e04d59b8e7"
server_key_iv: "ebda4d13fe01bed6edd4194daef55c42"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.361545 7069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "34b51efa70c449429918a62cc6d934c9"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "ecfae09fe54afe55f234f0e04d59b8e7"
server_key_iv: "ebda4d13fe01bed6edd4194daef55c42"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.365078 7069 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:00:27.367436 7099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.368609 7069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:27.368706 7069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "34b51efa70c449429918a62cc6d934c9"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "ecfae09fe54afe55f234f0e04d59b8e7"
server_key_iv: "ebda4d13fe01bed6edd4194daef55c42"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.368799 7069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:27.381115 7069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:27.383628 7069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:27.383783 7069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:27.384387 7069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:27.385143 7069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:27.385192 7069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.385236 7069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:27.385264 7069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.393565 7069 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:45401
I20260811 03:00:27.393594 7330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:45401 every 8 connection(s)
I20260811 03:00:27.394399 7069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:00:27.395718 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 7069
I20260811 03:00:27.395839 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:00:27.396173 9005 external_mini_cluster.cc:1428] Setting key c6d0cab5cf60d47fd81edaca677392cd
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417227, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 03:00:27.408087 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.396120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:43071 (local address 127.8.203.126:41919)
0811 03:00:27.396243 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.396248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.397200 (+ 952us) server_negotiation.cc:407] Connection header received
0811 03:00:27.398590 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.398593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.398647 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.398715 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:27.400028 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.400800 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.401921 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.402169 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.402256 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.404826 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:27.404829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:27.404831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:27.404855 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:27.406158 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.406639 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.406642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.406644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.406724 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.407031 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.407035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.407037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.407215 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:27.407298 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.407804 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.407921 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:00:27.408915 7334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.396618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:43071)
0811 03:00:27.397067 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.397108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.398388 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.398884 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.398891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.399260 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:27.399874 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.399892 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.400921 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.400924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.401817 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.401828 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.402330 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.402334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.402515 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.403067 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:27.403088 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:27.404667 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:27.406281 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:27.406288 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:27.406302 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:27.406547 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:27.406808 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:27.406811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:27.406813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:27.406942 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:27.407311 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:27.407318 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:27.407572 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.407746 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.408051 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 03:00:27.410051 7331 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41919
I20260811 03:00:27.410290 7331 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:27.410744 7331 heartbeater.cc:511] Master 127.8.203.126:41919 requested a full tablet report, sending...
I20260811 03:00:27.412283 6955 ts_manager.cc:194] Registered new tserver with Master: 34b51efa70c449429918a62cc6d934c9 (127.8.203.65:45401)
I20260811 03:00:27.413442 6955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:43071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 03:00:27.423884 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41919
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:27.532083 7338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:27.532323 7338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:27.532389 7338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:27.536294 7338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:27.536345 7338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:27.536427 7338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:00:27.541390 7338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:27.542330 7338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:27.543282 7338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:27.552371 7347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.552384 7353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.552446 7350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:27.552455 7338 server_base.cc:1034] running on GCE node
I20260811 03:00:27.553046 7338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:27.553675 7338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:27.554821 7338 hybrid_clock.cc:648] HybridClock initialized: now 1786417227554799 us; error 41 us; skew 500 ppm
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417227, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:27.557441 7338 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:00:27.558264 7338 webserver.cc:460] Webserver started at http://127.8.203.66:35417/ using document root <none> and password file <none>
I20260811 03:00:27.558842 7338 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:27.558911 7338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:27.559111 7338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:27.561060 7338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "7fd070f7db2943c2aed5b65f91526348"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "9318ea34def1123d8fd4d891725e18d4"
server_key_iv: "b5672844aa8b3528356b95300282351f"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.561710 7338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "7fd070f7db2943c2aed5b65f91526348"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "9318ea34def1123d8fd4d891725e18d4"
server_key_iv: "b5672844aa8b3528356b95300282351f"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.565219 7338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:00:27.567600 7367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.568722 7338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 03:00:27.568838 7338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "7fd070f7db2943c2aed5b65f91526348"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "9318ea34def1123d8fd4d891725e18d4"
server_key_iv: "b5672844aa8b3528356b95300282351f"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.568939 7338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:27.582906 7338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:27.585438 7338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:27.585572 7338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:27.586064 7338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:27.586794 7338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:27.586838 7338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.586881 7338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:27.586903 7338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.594885 7338 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:38373
I20260811 03:00:27.594913 7605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:38373 every 8 connection(s)
I20260811 03:00:27.595672 7338 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:00:27.599347 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 7338
I20260811 03:00:27.599458 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:00:27.599730 9005 external_mini_cluster.cc:1428] Setting key b932c01ef4db3817a5fef2bb587432fe
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417227, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:27.609361 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.597380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:54735 (local address 127.8.203.126:41919)
0811 03:00:27.597551 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.597557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.598423 (+ 866us) server_negotiation.cc:407] Connection header received
0811 03:00:27.600159 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.600162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.600222 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.600296 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:27.601383 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.601910 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.603084 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.603308 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.603399 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.606027 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:27.606030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:27.606032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:27.606066 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:27.607356 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.607928 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.607930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.607931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.607998 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.608334 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.608336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.608337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.608489 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:27.608561 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.609116 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.609208 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 03:00:27.610411 7610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.597784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:54735)
0811 03:00:27.598261 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.598314 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.599926 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.600474 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.600480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.600806 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:27.601243 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.601255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.602025 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.602028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.602957 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.602969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.603458 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.603462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.603644 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.604154 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:27.604174 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:27.605862 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:27.607471 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:27.607477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:27.607492 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:27.607841 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:27.608072 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:27.608076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:27.608078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:27.608257 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:27.608586 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:27.608593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:27.608885 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.609090 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.609441 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":108,"threads_started":1}
I20260811 03:00:27.611505 7606 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41919
I20260811 03:00:27.611697 7606 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:27.612108 7606 heartbeater.cc:511] Master 127.8.203.126:41919 requested a full tablet report, sending...
I20260811 03:00:27.613032 6955 ts_manager.cc:194] Registered new tserver with Master: 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
I20260811 03:00:27.613734 6955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:54735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 03:00:27.626134 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41919
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:27.734632 7618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:27.734854 7618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:27.734892 7618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:27.738718 7618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:27.738763 7618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:27.738816 7618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:00:27.743705 7618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:27.744568 7618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:27.745498 7618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:27.754859 7636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.754853 7631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:27.754853 7632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:27.755070 7618 server_base.cc:1034] running on GCE node
I20260811 03:00:27.755496 7618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:27.756085 7618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:27.757236 7618 hybrid_clock.cc:648] HybridClock initialized: now 1786417227757214 us; error 32 us; skew 500 ppm
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417227, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:27.759876 7618 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:00:27.760720 7618 webserver.cc:460] Webserver started at http://127.8.203.67:37565/ using document root <none> and password file <none>
I20260811 03:00:27.761219 7618 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:27.761273 7618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:27.761498 7618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:00:27.763439 7618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "692e9d80bb0c4c3085a9773ced2a2076"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "b0587ec45ac6198e73d7a1053a0cd1fa"
server_key_iv: "d6c911b3c57ceeda0264249556fafea4"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.763957 7618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "692e9d80bb0c4c3085a9773ced2a2076"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "b0587ec45ac6198e73d7a1053a0cd1fa"
server_key_iv: "d6c911b3c57ceeda0264249556fafea4"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.767586 7618 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 03:00:27.769881 7649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.771030 7618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 03:00:27.771131 7618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "692e9d80bb0c4c3085a9773ced2a2076"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "b0587ec45ac6198e73d7a1053a0cd1fa"
server_key_iv: "d6c911b3c57ceeda0264249556fafea4"
server_key_version: "encryptionkey@0"
I20260811 03:00:27.771219 7618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:27.785189 7618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:27.788062 7618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:27.788220 7618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:27.788746 7618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:27.789528 7618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:27.789577 7618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.789629 7618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:27.789654 7618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:27.797827 7618 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43947
I20260811 03:00:27.797854 7878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43947 every 8 connection(s)
I20260811 03:00:27.798635 7618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:00:27.801365 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 7618
I20260811 03:00:27.801476 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:00:27.801776 9005 external_mini_cluster.cc:1428] Setting key 9a7254ee70ec33a459fd8b2f1026fbd0
Aug 11 03:00:27 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417227, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:27.810798 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.800255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:60661 (local address 127.8.203.126:41919)
0811 03:00:27.800400 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.800405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.801205 (+ 800us) server_negotiation.cc:407] Connection header received
0811 03:00:27.802732 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.802735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.802782 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.802842 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:27.804152 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.804665 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.805742 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.805943 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.806010 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.808129 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:27.808131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:27.808133 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:27.808153 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:27.809215 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.809667 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.809670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.809671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.809733 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.809985 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.809987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.809988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.810136 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:27.810205 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.810597 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.810684 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 03:00:27.811522 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.800623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:60661)
0811 03:00:27.801049 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.801095 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.802550 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.803015 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.803022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.803441 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:27.804022 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.804039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.804749 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.804751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.805657 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.805667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.806044 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.806047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.806178 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.806675 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:27.806691 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:27.807999 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:27.809349 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:27.809354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:27.809365 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:27.809599 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:27.809794 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:27.809795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:27.809797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:27.809919 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:27.810205 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:27.810208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:27.810420 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.810558 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.810782 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":117,"threads_started":1}
I20260811 03:00:27.812465 7879 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41919
I20260811 03:00:27.812674 7879 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:27.813100 7879 heartbeater.cc:511] Master 127.8.203.126:41919 requested a full tablet report, sending...
I20260811 03:00:27.814033 6955 ts_manager.cc:194] Registered new tserver with Master: 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947)
I20260811 03:00:27.814640 6955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:60661
I20260811 03:00:27.815420 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:00:27.823341 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.817509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.8.203.126:41919)
0811 03:00:27.817615 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.817620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.817803 (+ 183us) server_negotiation.cc:407] Connection header received
0811 03:00:27.817908 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.817910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.817943 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.817993 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:27.818541 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.818997 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.819698 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.819882 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.819942 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.820974 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:27.820977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:27.820978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:27.820997 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:27.822176 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.822485 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.822487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.822489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.822550 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:27.822720 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:27.822721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:27.822722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:27.822898 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:27.822989 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.823129 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.823209 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 03:00:27.825651 6955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:00:27.828420 6955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:00:27.841863 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.835994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43947 (local address 127.0.0.1:43624)
0811 03:00:27.836344 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.836394 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.836512 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.837444 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.837450 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.837475 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:27.837601 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.837609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.838979 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.838982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.840389 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.840399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.841463 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.841467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.841551 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.841576 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.841621 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.841719 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":111,"thread_start_us":85,"threads_started":1}
I20260811 03:00:27.842083 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.836629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35442 (local address 127.8.203.65:45401)
0811 03:00:27.837346 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.837358 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.837381 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:00:27.837454 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.837460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.837643 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.837801 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:27.837986 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.838796 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.840702 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.841489 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.841584 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.841721 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.841850 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.841942 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":116,"threads_started":1}
I20260811 03:00:27.842180 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.836864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41208 (local address 127.8.203.66:38373)
0811 03:00:27.837235 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.837245 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.837634 (+ 389us) server_negotiation.cc:407] Connection header received
0811 03:00:27.837717 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.837722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.837888 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.838022 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:27.838406 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.839297 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.840598 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.841166 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.841262 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.841426 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.841727 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.841825 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":99,"threads_started":1}
I20260811 03:00:27.842419 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.836382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45401 (local address 127.0.0.1:35442)
0811 03:00:27.836983 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.836996 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.837077 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.837794 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.837797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.837813 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:27.837904 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.837912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.838913 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.838916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.840591 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.840600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.841600 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.841603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.841724 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.841745 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.841815 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.841874 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":320,"thread_start_us":109,"threads_started":1}
I20260811 03:00:27.842429 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.836144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.8.203.67:43947)
0811 03:00:27.836941 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.836954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.836978 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:00:27.837049 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.837054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.837247 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.837728 (+ 481us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:27.837877 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.838790 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.840512 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.841282 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.842040 (+ 758us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.842181 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.842232 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.842315 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":123,"threads_started":1}
I20260811 03:00:27.843353 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.836678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38373 (local address 127.0.0.1:41208)
0811 03:00:27.837450 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.837463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.837546 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.838175 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.838178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.838196 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:27.838275 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.838280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.839421 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.839425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.840472 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.840485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.841275 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.841279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.841420 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.841462 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.841568 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.843233 (+ 1665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":1582,"thread_start_us":82,"threads_started":1}
I20260811 03:00:27.845014 7732 tablet_service.cc:1467] Processing CreateTablet for tablet a4c32c215721469ba832473252ff5a90 (DEFAULT_TABLE table=test-table [id=895dce160dc14225aec05b33aa1410e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:27.845160 7200 tablet_service.cc:1467] Processing CreateTablet for tablet a4c32c215721469ba832473252ff5a90 (DEFAULT_TABLE table=test-table [id=895dce160dc14225aec05b33aa1410e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:27.845952 7732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c32c215721469ba832473252ff5a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:27.845803 7454 tablet_service.cc:1467] Processing CreateTablet for tablet a4c32c215721469ba832473252ff5a90 (DEFAULT_TABLE table=test-table [id=895dce160dc14225aec05b33aa1410e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:27.846218 7200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c32c215721469ba832473252ff5a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:27.846761 7454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4c32c215721469ba832473252ff5a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:27.852205 7919 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Bootstrap starting.
I20260811 03:00:27.853636 7923 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Bootstrap starting.
I20260811 03:00:27.853947 7922 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Bootstrap starting.
I20260811 03:00:27.854580 7919 tablet_bootstrap.cc:654] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:27.855901 7919 log.cc:824] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:27.856199 7922 tablet_bootstrap.cc:654] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:27.856205 7923 tablet_bootstrap.cc:654] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:27.856925 7922 log.cc:824] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:27.857031 7923 log.cc:824] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:27.858090 7919 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: No bootstrap required, opened a new log
I20260811 03:00:27.858280 7919 ts_tablet_manager.cc:1397] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 03:00:27.858736 7923 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: No bootstrap required, opened a new log
I20260811 03:00:27.858909 7923 ts_tablet_manager.cc:1397] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 03:00:27.858958 7922 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: No bootstrap required, opened a new log
I20260811 03:00:27.859129 7922 ts_tablet_manager.cc:1397] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:00:27.861647 7923 raft_consensus.cc:348] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.861904 7923 raft_consensus.cc:374] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:27.861965 7923 raft_consensus.cc:729] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd070f7db2943c2aed5b65f91526348, State: Initialized, Role: FOLLOWER
I20260811 03:00:27.862102 7919 raft_consensus.cc:348] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.862382 7919 raft_consensus.cc:374] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:27.862445 7919 raft_consensus.cc:729] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 692e9d80bb0c4c3085a9773ced2a2076, State: Initialized, Role: FOLLOWER
I20260811 03:00:27.862489 7923 consensus_queue.cc:260] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.862869 7919 consensus_queue.cc:260] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.862751 7922 raft_consensus.cc:348] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.863016 7922 raft_consensus.cc:374] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:27.863076 7922 raft_consensus.cc:729] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b51efa70c449429918a62cc6d934c9, State: Initialized, Role: FOLLOWER
I20260811 03:00:27.863351 7606 heartbeater.cc:503] Master 127.8.203.126:41919 was elected leader, sending a full tablet report...
I20260811 03:00:27.863528 7922 consensus_queue.cc:260] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.863886 7919 ts_tablet_manager.cc:1428] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 03:00:27.864377 7923 ts_tablet_manager.cc:1428] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260811 03:00:27.864387 7922 ts_tablet_manager.cc:1428] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 03:00:27.864403 7879 heartbeater.cc:503] Master 127.8.203.126:41919 was elected leader, sending a full tablet report...
I20260811 03:00:27.864689 7331 heartbeater.cc:503] Master 127.8.203.126:41919 was elected leader, sending a full tablet report...
W20260811 03:00:27.895452 7333 tablet.cc:2369] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:00:27.930914 7929 raft_consensus.cc:482] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:27.931131 7929 raft_consensus.cc:504] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.932210 7929 leader_election.cc:288] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947), 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
I20260811 03:00:27.936888 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.932552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:40105 (local address 127.8.203.67:43947)
0811 03:00:27.932700 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.932705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.932813 (+ 108us) server_negotiation.cc:407] Connection header received
0811 03:00:27.932984 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.932988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.933042 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.933120 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:27.933387 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.934132 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.935657 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.936274 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.936397 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.936451 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.936693 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.936756 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 03:00:27.936888 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.933012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:34685 (local address 127.8.203.66:38373)
0811 03:00:27.933157 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.933162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.933177 (+ 15us) server_negotiation.cc:407] Connection header received
0811 03:00:27.933229 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.933232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.933272 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.933364 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:27.933600 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.934289 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.935645 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.936274 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.936397 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.936451 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.936688 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.936756 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:00:27.936942 7932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.932438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43947 (local address 127.8.203.65:40105)
0811 03:00:27.932735 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.932748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.932855 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.933145 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.933148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.933162 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:27.933284 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.933304 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.934232 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.934235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.935531 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.935542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.936384 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.936388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.936571 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.936605 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.936662 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.936762 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":81,"thread_start_us":116,"threads_started":1}
I20260811 03:00:27.936951 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.932832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38373 (local address 127.8.203.65:34685)
0811 03:00:27.933051 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:27.933062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:27.933143 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:27.933392 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:27.933395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:27.933411 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:27.933502 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.933506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.934385 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.934388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.935531 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:27.935542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.936397 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.936400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:27.936516 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.936592 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 03:00:27.936666 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 03:00:27.936762 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":51,"thread_start_us":67,"threads_started":1}
I20260811 03:00:27.937639 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c32c215721469ba832473252ff5a90" candidate_uuid: "34b51efa70c449429918a62cc6d934c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "692e9d80bb0c4c3085a9773ced2a2076" is_pre_election: true
I20260811 03:00:27.937637 7499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c32c215721469ba832473252ff5a90" candidate_uuid: "34b51efa70c449429918a62cc6d934c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd070f7db2943c2aed5b65f91526348" is_pre_election: true
I20260811 03:00:27.937906 7775 raft_consensus.cc:2454] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b51efa70c449429918a62cc6d934c9 in term 0.
I20260811 03:00:27.937923 7499 raft_consensus.cc:2454] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b51efa70c449429918a62cc6d934c9 in term 0.
I20260811 03:00:27.938335 7104 leader_election.cc:302] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9, 7fd070f7db2943c2aed5b65f91526348; no voters:
I20260811 03:00:27.938586 7929 raft_consensus.cc:2781] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:00:27.938665 7929 raft_consensus.cc:482] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:00:27.938717 7929 raft_consensus.cc:3037] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:27.939721 7929 raft_consensus.cc:504] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.940019 7929 leader_election.cc:288] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 1 election: Requested vote from peers 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947), 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
I20260811 03:00:27.940349 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c32c215721469ba832473252ff5a90" candidate_uuid: "34b51efa70c449429918a62cc6d934c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "692e9d80bb0c4c3085a9773ced2a2076"
I20260811 03:00:27.940402 7499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c32c215721469ba832473252ff5a90" candidate_uuid: "34b51efa70c449429918a62cc6d934c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd070f7db2943c2aed5b65f91526348"
I20260811 03:00:27.940491 7499 raft_consensus.cc:3037] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:27.940495 7775 raft_consensus.cc:3037] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:27.941504 7775 raft_consensus.cc:2454] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b51efa70c449429918a62cc6d934c9 in term 1.
I20260811 03:00:27.941507 7499 raft_consensus.cc:2454] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b51efa70c449429918a62cc6d934c9 in term 1.
I20260811 03:00:27.941777 7104 leader_election.cc:302] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9, 692e9d80bb0c4c3085a9773ced2a2076; no voters:
I20260811 03:00:27.941949 7929 raft_consensus.cc:2781] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:27.942163 7929 raft_consensus.cc:686] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 1 LEADER]: Becoming Leader. State: Replica: 34b51efa70c449429918a62cc6d934c9, State: Running, Role: LEADER
I20260811 03:00:27.942463 7929 consensus_queue.cc:237] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:27.945546 6952 catalog_manager.cc:5495] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34b51efa70c449429918a62cc6d934c9 (127.8.203.65). New cstate: current_term: 1 leader_uuid: "34b51efa70c449429918a62cc6d934c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } health_report { overall_health: HEALTHY } } }
I20260811 03:00:27.960860 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:27.957576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.8.203.65:45401)
0811 03:00:27.957680 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:27.957684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:27.957696 (+ 12us) server_negotiation.cc:407] Connection header received
0811 03:00:27.957856 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:27.957858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:27.957917 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:27.958001 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:00:27.958240 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.958755 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.959581 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:27.959783 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:27.959867 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:27.960146 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:00:27.960520 (+ 374us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:00:27.960621 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:27.960689 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 03:00:27.960740 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 03:00:27.966248 7499 raft_consensus.cc:1261] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Refusing update from remote peer 34b51efa70c449429918a62cc6d934c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:27.966259 7775 raft_consensus.cc:1261] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 FOLLOWER]: Refusing update from remote peer 34b51efa70c449429918a62cc6d934c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:27.966873 7937 consensus_queue.cc:1035] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:27.967103 7929 consensus_queue.cc:1035] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:27.972290 7944 mvcc.cc:205] Tried to move back new op lower bound from 7317164965743128576 to 7317164965654646784. Current Snapshot: MvccSnapshot[applied={T|T < 7317164965743128576}]
I20260811 03:00:27.980927 6952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:27.981052 6952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:27.983235 6952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Sending DeleteTablet for 3 replicas of tablet a4c32c215721469ba832473252ff5a90
I20260811 03:00:27.983916 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 7069
I20260811 03:00:27.984004 7200 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c32c215721469ba832473252ff5a90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:27 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:35442
I20260811 03:00:27.987015 7732 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c32c215721469ba832473252ff5a90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:27 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:43624
I20260811 03:00:27.995031 7454 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c32c215721469ba832473252ff5a90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:27 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:41208
I20260811 03:00:27.995095 7968 tablet_replica.cc:330] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: stopping tablet replica
I20260811 03:00:27.995396 7968 raft_consensus.cc:2229] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:27.995499 7970 tablet_replica.cc:330] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: stopping tablet replica
I20260811 03:00:27.995721 7968 raft_consensus.cc:2258] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:27.995772 7970 raft_consensus.cc:2229] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:27.996055 7970 raft_consensus.cc:2258] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:27.997332 7968 ts_tablet_manager.cc:1905] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:27.997341 7970 ts_tablet_manager.cc:1905] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:27.999954 7970 ts_tablet_manager.cc:1918] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:27.999954 7968 ts_tablet_manager.cc:1918] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:28.000057 7968 log.cc:1196] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/a4c32c215721469ba832473252ff5a90
I20260811 03:00:28.000057 7970 log.cc:1196] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/a4c32c215721469ba832473252ff5a90
I20260811 03:00:28.000401 7970 ts_tablet_manager.cc:1939] T a4c32c215721469ba832473252ff5a90 P 7fd070f7db2943c2aed5b65f91526348: Deleting consensus metadata
I20260811 03:00:28.000401 7968 ts_tablet_manager.cc:1939] T a4c32c215721469ba832473252ff5a90 P 692e9d80bb0c4c3085a9773ced2a2076: Deleting consensus metadata
I20260811 03:00:28.001647 6938 catalog_manager.cc:4841] TS 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373): tablet a4c32c215721469ba832473252ff5a90 (table test-table [id=895dce160dc14225aec05b33aa1410e6]) successfully deleted
I20260811 03:00:28.001829 6938 catalog_manager.cc:4841] TS 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947): tablet a4c32c215721469ba832473252ff5a90 (table test-table [id=895dce160dc14225aec05b33aa1410e6]) successfully deleted
W20260811 03:00:28.005069 6934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:45401 (error 108)
I20260811 03:00:28.005308 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 7338
I20260811 03:00:28.008131 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:28.007575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45401 (local address 127.0.0.1:35462)
0811 03:00:28.007956 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:28.008038 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292}
W20260811 03:00:28.010083 6934 catalog_manager.cc:4584] TS 34b51efa70c449429918a62cc6d934c9 (127.8.203.65:45401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4c32c215721469ba832473252ff5a90: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45401: connect: Connection refused (error 111)
I20260811 03:00:28.016574 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 7618
I20260811 03:00:28.034348 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 6904
I20260811 03:00:48.053467 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41919
--webserver_interface=127.8.203.126
--webserver_port=38175
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:41919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:00:48.165544 7972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:48.165788 7972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:48.165825 7972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:48.169658 7972 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 03:00:48.169701 7972 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 03:00:48.169716 7972 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 03:00:48.169728 7972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:00:48.169750 7972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:48.169772 7972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:48.169791 7972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:00:48.169806 7972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:00:48.174866 7972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:41919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:48.175796 7972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:48.176769 7972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:48.190361 7982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:48.190409 7988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:48.190438 7972 server_base.cc:1034] running on GCE node
W20260811 03:00:48.190366 7984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:48.191165 7972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:48.192257 7972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:48.193410 7972 hybrid_clock.cc:648] HybridClock initialized: now 1786417248193391 us; error 31 us; skew 500 ppm
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:48.199186 7972 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:00:48.200052 7972 webserver.cc:460] Webserver started at http://127.8.203.126:38175/ using document root <none> and password file <none>
I20260811 03:00:48.200606 7972 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:48.200690 7972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:48.204447 7972 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 03:00:48.206524 8002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.207686 7972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:00:48.207799 7972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "d1829f20ad1e467daf847875f99a238a"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "d5308f5aaa5ec21226af834f297826a3"
server_key_iv: "cbfeb25ec2bb2d3ac61cae667aea17b5"
server_key_version: "encryptionkey@0"
I20260811 03:00:48.208384 7972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:48.227473 7972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:48.230789 7972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:48.230989 7972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:48.238345 7972 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:41919
I20260811 03:00:48.238590 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:41919 every 8 connection(s)
I20260811 03:00:48.239498 7972 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:00:48.240679 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 7972
I20260811 03:00:48.241163 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:45401
--local_ip_for_outbound_sockets=127.8.203.65
--tserver_master_addrs=127.8.203.126:41919
--webserver_port=33109
--webserver_interface=127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:48.244560 8103 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:00:48.246273 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Bootstrap starting.
I20260811 03:00:48.262993 8103 log.cc:824] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:48.271507 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:00:48.271823 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Bootstrap complete.
I20260811 03:00:48.275615 8103 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:48.276005 8103 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1829f20ad1e467daf847875f99a238a, State: Initialized, Role: FOLLOWER
I20260811 03:00:48.276492 8103 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:48.276611 8103 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:00:48.276679 8103 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:00:48.276767 8103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 1 FOLLOWER]: Advancing to term 2
I20260811 03:00:48.278609 8103 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:48.278915 8103 leader_election.cc:302] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1829f20ad1e467daf847875f99a238a; no voters:
I20260811 03:00:48.279187 8103 leader_election.cc:288] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 03:00:48.279280 8110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 03:00:48.279476 8110 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 LEADER]: Becoming Leader. State: Replica: d1829f20ad1e467daf847875f99a238a, State: Running, Role: LEADER
I20260811 03:00:48.279815 8110 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:00:48.280432 8103 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:00:48.280779 8110 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1829f20ad1e467daf847875f99a238a. Latest consensus state: current_term: 2 leader_uuid: "d1829f20ad1e467daf847875f99a238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } } }
I20260811 03:00:48.280905 8110 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:48.281275 8117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:00:48.281323 8112 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d1829f20ad1e467daf847875f99a238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } } }
I20260811 03:00:48.281505 8112 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: This master's current role is: LEADER
I20260811 03:00:48.286139 8117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:00:48.286865 8117 catalog_manager.cc:1259] Loaded cluster ID: a4d008e7c9a94f238c185328caa038af
I20260811 03:00:48.286928 8117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:00:48.289366 8117 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:00:48.290066 8117 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Loaded TSK: 0
I20260811 03:00:48.305169 8117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Generated new TSK 1
I20260811 03:00:48.305442 8117 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:00:48.354023 8107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:48.354280 8107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:48.354348 8107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:48.358294 8107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:48.358352 8107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:48.358434 8107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:00:48.363567 8107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:45401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=33109
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:48.364554 8107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:48.365550 8107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:48.375847 8148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:48.375854 8149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:48.375900 8154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:48.376278 8107 server_base.cc:1034] running on GCE node
I20260811 03:00:48.376655 8107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:48.377238 8107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:48.378414 8107 hybrid_clock.cc:648] HybridClock initialized: now 1786417248378391 us; error 40 us; skew 500 ppm
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:48.381283 8107 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:00:48.382356 8107 webserver.cc:460] Webserver started at http://127.8.203.65:33109/ using document root <none> and password file <none>
I20260811 03:00:48.383034 8107 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:48.383136 8107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:48.387982 8107 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:00:48.390596 8174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.392059 8107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 03:00:48.392186 8107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "34b51efa70c449429918a62cc6d934c9"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "ecfae09fe54afe55f234f0e04d59b8e7"
server_key_iv: "ebda4d13fe01bed6edd4194daef55c42"
server_key_version: "encryptionkey@0"
I20260811 03:00:48.392866 8107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:48.413925 8107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:48.417181 8107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:48.417426 8107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:48.418104 8107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:48.419335 8189 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 03:00:48.423317 8107 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 03:00:48.423388 8107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.001s
I20260811 03:00:48.423447 8107 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 03:00:48.425554 8107 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 03:00:48.425606 8107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.001s
I20260811 03:00:48.425865 8189 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Bootstrap starting.
I20260811 03:00:48.435526 8107 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:45401
I20260811 03:00:48.435858 8383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:45401 every 8 connection(s)
I20260811 03:00:48.436581 8107 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:00:48.438421 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 8107
I20260811 03:00:48.438891 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:38373
--local_ip_for_outbound_sockets=127.8.203.66
--tserver_master_addrs=127.8.203.126:41919
--webserver_port=35417
--webserver_interface=127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:48.445561 8189 log.cc:824] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:48.453521 8189 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:00:48.453930 8189 tablet_bootstrap.cc:492] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Bootstrap complete.
I20260811 03:00:48.454248 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.438686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:52039)
0811 03:00:48.439517 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:48.439570 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:48.440323 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:48.441625 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:48.441636 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:48.442143 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:48.442776 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.442796 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.444134 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.444136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.445023 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.445038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.445803 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.445807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.446011 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.446646 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:48.446667 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:48.448491 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:48.450880 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.450887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.450902 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.451249 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.451658 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.451661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.451663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.451843 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.452509 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:48.452515 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:48.452797 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 03:00:48.452993 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 03:00:48.453337 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":114,"threads_started":1}
I20260811 03:00:48.454702 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.438683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:52039 (local address 127.8.203.126:41919)
0811 03:00:48.439239 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.439260 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.439676 (+ 416us) server_negotiation.cc:407] Connection header received
0811 03:00:48.440721 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.440737 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.441128 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.441501 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:48.442978 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.443965 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.445152 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.445648 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.445807 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.448701 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:48.448708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:48.448724 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:48.448756 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:48.450737 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.451375 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.451381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.451386 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.451542 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.451952 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.451955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.451957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.452380 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:48.452503 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.453138 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.453461 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":111,"threads_started":1}
I20260811 03:00:48.455150 8189 ts_tablet_manager.cc:1397] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Time spent bootstrapping tablet: real 0.030s user 0.029s sys 0.000s
I20260811 03:00:48.457204 8384 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41919
I20260811 03:00:48.457453 8384 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:48.457988 8384 heartbeater.cc:511] Master 127.8.203.126:41919 requested a full tablet report, sending...
I20260811 03:00:48.457612 8189 raft_consensus.cc:348] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.458081 8189 raft_consensus.cc:729] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b51efa70c449429918a62cc6d934c9, State: Initialized, Role: FOLLOWER
I20260811 03:00:48.458549 8189 consensus_queue.cc:260] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.459502 8189 ts_tablet_manager.cc:1428] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 03:00:48.461266 8032 ts_manager.cc:194] Registered new tserver with Master: 34b51efa70c449429918a62cc6d934c9 (127.8.203.65:45401)
I20260811 03:00:48.465051 8032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:52039
I20260811 03:00:48.467418 8384 heartbeater.cc:503] Master 127.8.203.126:41919 was elected leader, sending a full tablet report...
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.65@KRBTEST.COM
I20260811 03:00:48.474768 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.464462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59504 (local address 127.8.203.65:45401)
0811 03:00:48.464819 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.464830 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.465352 (+ 522us) server_negotiation.cc:407] Connection header received
0811 03:00:48.465476 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.465482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.465633 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.465749 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:48.466573 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.467650 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.468834 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.469343 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.469447 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.471664 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:48.471672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:48.471682 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:48.471703 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:48.473041 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.473578 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.473582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.473583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.473662 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.473922 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.473924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.473925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.474123 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:48.474216 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.474530 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.474653 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":92,"threads_started":1}
I20260811 03:00:48.474810 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.464401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45401 (local address 127.0.0.1:59504)
0811 03:00:48.465069 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:48.465102 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:48.465227 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:48.465911 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:48.465918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:48.466321 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:48.466448 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.466456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.467754 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.467758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.468592 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.468604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.469450 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.469453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.469534 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.470007 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:48.470019 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:48.471538 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:48.473145 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.473154 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.473159 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.473484 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.473743 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.473746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.473747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.473840 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.474215 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:48.474221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:48.474337 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 03:00:48.474478 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 03:00:48.474653 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":113,"threads_started":1}
I20260811 03:00:48.475747 8268 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c32c215721469ba832473252ff5a90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:27 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:59504
I20260811 03:00:48.476547 8404 tablet_replica.cc:330] stopping tablet replica
I20260811 03:00:48.476850 8404 raft_consensus.cc:2229] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:48.477106 8404 raft_consensus.cc:2258] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:48.478250 8404 ts_tablet_manager.cc:1905] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:48.482771 8404 ts_tablet_manager.cc:1918] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:48.482880 8404 log.cc:1196] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal/wals/a4c32c215721469ba832473252ff5a90
I20260811 03:00:48.483136 8404 ts_tablet_manager.cc:1939] T a4c32c215721469ba832473252ff5a90 P 34b51efa70c449429918a62cc6d934c9: Deleting consensus metadata
I20260811 03:00:48.485006 8007 catalog_manager.cc:4841] TS 34b51efa70c449429918a62cc6d934c9 (127.8.203.65:45401): tablet a4c32c215721469ba832473252ff5a90 (table test-table [id=895dce160dc14225aec05b33aa1410e6]) successfully deleted
W20260811 03:00:48.560686 8389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:48.560920 8389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:48.560959 8389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:48.564803 8389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:48.564851 8389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:48.564905 8389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:00:48.569952 8389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:38373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=35417
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:48.570906 8389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:48.572023 8389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:48.581657 8412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:48.581652 8417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:48.581652 8410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:48.581662 8389 server_base.cc:1034] running on GCE node
I20260811 03:00:48.582319 8389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:48.582899 8389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:48.584048 8389 hybrid_clock.cc:648] HybridClock initialized: now 1786417248584025 us; error 41 us; skew 500 ppm
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:48.586544 8389 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:00:48.587378 8389 webserver.cc:460] Webserver started at http://127.8.203.66:35417/ using document root <none> and password file <none>
I20260811 03:00:48.587898 8389 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:48.587984 8389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:48.591773 8389 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 03:00:48.593724 8428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.594856 8389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:48.594960 8389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "7fd070f7db2943c2aed5b65f91526348"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "9318ea34def1123d8fd4d891725e18d4"
server_key_iv: "b5672844aa8b3528356b95300282351f"
server_key_version: "encryptionkey@0"
I20260811 03:00:48.595475 8389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:48.605115 8389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:48.607620 8389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:48.607776 8389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:48.608297 8389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:48.609062 8389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:48.609108 8389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.609158 8389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:48.609187 8389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.617564 8389 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:38373
I20260811 03:00:48.617616 8649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:38373 every 8 connection(s)
I20260811 03:00:48.618503 8389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:48.627631 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 8389
I20260811 03:00:48.628086 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:43947
--local_ip_for_outbound_sockets=127.8.203.67
--tserver_master_addrs=127.8.203.126:41919
--webserver_port=37565
--webserver_interface=127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:00:48.630851 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.620116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:46671 (local address 127.8.203.126:41919)
0811 03:00:48.620250 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.620254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.621046 (+ 792us) server_negotiation.cc:407] Connection header received
0811 03:00:48.622385 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.622389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.622440 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.622511 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:48.623555 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.624066 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.625071 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.625284 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.625398 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.627541 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:48.627544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:48.627546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:48.627570 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:48.628911 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.629406 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.629410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.629411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.629491 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.629808 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.629811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.629813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.629997 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:48.630081 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.630548 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.630679 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:00:48.631567 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.620470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:46671)
0811 03:00:48.620892 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:48.620933 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:48.622197 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:48.622657 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:48.622663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:48.622986 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:48.623447 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.623460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.624151 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.624153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.624985 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.624993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.625427 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.625429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.625567 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.626027 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:48.626045 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:48.627380 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:48.629022 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.629028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.629042 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.629283 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.629575 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.629578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.629580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.629719 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.630095 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:48.630102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:48.630318 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 03:00:48.630486 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 03:00:48.630791 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":119,"threads_started":1}
I20260811 03:00:48.632633 8650 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41919
I20260811 03:00:48.632858 8650 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:48.633320 8650 heartbeater.cc:511] Master 127.8.203.126:41919 requested a full tablet report, sending...
I20260811 03:00:48.634330 8032 ts_manager.cc:194] Registered new tserver with Master: 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
I20260811 03:00:48.635070 8032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:46671
W20260811 03:00:48.748378 8654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:00:48.748600 8654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:00:48.748638 8654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:00:48.752478 8654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:00:48.752530 8654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:00:48.752583 8654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:00:48.757536 8654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:43947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=37565
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:00:48.758420 8654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:00:48.759325 8654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:00:48.768260 8663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:00:48.768272 8664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:48.768290 8654 server_base.cc:1034] running on GCE node
W20260811 03:00:48.768479 8668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:00:48.768817 8654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:00:48.769426 8654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:00:48.770582 8654 hybrid_clock.cc:648] HybridClock initialized: now 1786417248770561 us; error 31 us; skew 500 ppm
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:00:48.773027 8654 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:00:48.773890 8654 webserver.cc:460] Webserver started at http://127.8.203.67:37565/ using document root <none> and password file <none>
I20260811 03:00:48.774384 8654 fs_manager.cc:359] Metadata directory not provided
I20260811 03:00:48.774453 8654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:00:48.777983 8654 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:00:48.779908 8680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.781077 8654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:00:48.781209 8654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "692e9d80bb0c4c3085a9773ced2a2076"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "b0587ec45ac6198e73d7a1053a0cd1fa"
server_key_iv: "d6c911b3c57ceeda0264249556fafea4"
server_key_version: "encryptionkey@0"
I20260811 03:00:48.781776 8654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:00:48.792625 8654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:00:48.795028 8654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:00:48.795179 8654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:00:48.795682 8654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:00:48.796427 8654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:00:48.796473 8654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.796522 8654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:00:48.796550 8654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:00:48.805058 8654 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43947
I20260811 03:00:48.805117 8900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43947 every 8 connection(s)
I20260811 03:00:48.806001 8654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:00:48.808804 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 8654
Aug 11 03:00:48 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:00:48.817739 8915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.809399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44508 (local address 127.8.203.126:41919)
0811 03:00:48.809615 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.809619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.809873 (+ 254us) server_negotiation.cc:407] Connection header received
0811 03:00:48.809990 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.809993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.810041 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.810132 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:48.810825 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.811537 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.812723 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.812984 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.813059 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.814493 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:48.814497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:48.814500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:48.814524 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:48.815985 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.816453 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.816457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.816459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.816569 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.816859 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.816862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.816863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.817068 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:48.817319 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.817468 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.817590 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260811 03:00:48.819696 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.807649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:36553 (local address 127.8.203.126:41919)
0811 03:00:48.807770 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.807773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.808530 (+ 757us) server_negotiation.cc:407] Connection header received
0811 03:00:48.809936 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.809941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.809994 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.810059 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:48.811298 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.811855 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.813324 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.813540 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.813625 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.816163 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:48.816167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:48.816169 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:48.816196 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:48.817806 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.818379 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.818383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.818385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.818459 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.818759 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.818762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.818764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.818922 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:48.819003 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.819452 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.819557 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 03:00:48.820430 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.807990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:36553)
0811 03:00:48.808401 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:48.808441 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:48.809732 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:48.810253 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:48.810260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:48.810594 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:00:48.811159 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.811177 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.811968 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.811972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.813191 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.813201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.813679 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.813684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.813826 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.814296 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:00:48.814311 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:00:48.816011 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:00:48.817946 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.817951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.817965 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.818260 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.818540 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:00:48.818543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:00:48.818545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:00:48.818673 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:00:48.819020 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:00:48.819026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:00:48.819237 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 03:00:48.819397 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 03:00:48.819682 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":109,"threads_started":1}
I20260811 03:00:48.821455 8904 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41919
I20260811 03:00:48.821672 8904 heartbeater.cc:464] Registering TS with master...
I20260811 03:00:48.822089 8904 heartbeater.cc:511] Master 127.8.203.126:41919 requested a full tablet report, sending...
I20260811 03:00:48.822983 8032 ts_manager.cc:194] Registered new tserver with Master: 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947)
I20260811 03:00:48.823483 8032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:36553
I20260811 03:00:48.830577 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:00:48.833957 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.831123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44520 (local address 127.8.203.126:41919)
0811 03:00:48.831218 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.831221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.831387 (+ 166us) server_negotiation.cc:407] Connection header received
0811 03:00:48.831501 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.831503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.831582 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.831654 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:00:48.831787 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.832241 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.833067 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.833266 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.833378 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.833589 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:00:48.833696 (+ 107us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 03:00:48.833875 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44520: token expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 03:00:48.834007 8387 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44520: token expired
W20260811 03:00:48.834395 7884 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.203.126:41919)
I20260811 03:00:48.839995 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.834691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.8.203.126:41919)
0811 03:00:48.834781 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.834784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.834794 (+ 10us) server_negotiation.cc:407] Connection header received
0811 03:00:48.834873 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.834875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.834907 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.834962 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:00:48.835426 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.835861 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.836524 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.836703 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.836763 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.837849 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:00:48.837852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:00:48.837854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:00:48.837875 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:00:48.838907 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.839234 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.839235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.839237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.839295 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:00:48.839462 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:00:48.839464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:00:48.839465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:00:48.839595 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:00:48.839682 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.839816 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.839902 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 03:00:48.841950 8032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:00:48.844934 8032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:00:48.851907 8268 tablet_service.cc:1467] Processing CreateTablet for tablet 8fabb6959ccc4b0baaa582a9ba2f1496 (DEFAULT_TABLE table=test-table [id=2220e10b387b4332a3866e7fe9d21bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:48.852517 8268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fabb6959ccc4b0baaa582a9ba2f1496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:48.855298 8189 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Bootstrap starting.
I20260811 03:00:48.855872 8927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.851123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38373 (local address 127.0.0.1:33460)
0811 03:00:48.851576 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:48.851586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:48.851652 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:48.852260 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:48.852263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:48.852282 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:48.852364 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.852370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.853693 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.853696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.854612 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.854622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.855471 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.855475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.855560 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.855605 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 03:00:48.855680 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 03:00:48.855746 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":96,"threads_started":1}
I20260811 03:00:48.856228 8189 tablet_bootstrap.cc:654] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:48.856292 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.851554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.8.203.66:38373)
0811 03:00:48.851849 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.851857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.851884 (+ 27us) server_negotiation.cc:407] Connection header received
0811 03:00:48.851943 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.851947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.852120 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.852267 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:48.852468 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.853567 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.854732 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.855297 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.855411 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.855550 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.856047 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.856145 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":101,"threads_started":1}
I20260811 03:00:48.857357 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.850846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43947 (local address 127.0.0.1:48038)
0811 03:00:48.850965 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:48.850979 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:48.851051 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:48.852479 (+ 1428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:48.852482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:48.852501 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:48.852574 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.852580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.854190 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.854194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.855253 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:48.855265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.856949 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.856953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:48.857046 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.857067 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 03:00:48.857139 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 03:00:48.857213 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 03:00:48.857630 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:48.850990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.8.203.67:43947)
0811 03:00:48.851397 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:48.851406 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:48.851424 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:00:48.852129 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:48.852135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:48.852318 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:48.852716 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:48.852857 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.854066 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.855458 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:48.856823 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:48.856931 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:48.857052 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:48.857410 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 03:00:48.857502 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":147,"threads_started":1}
I20260811 03:00:48.857688 8189 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: No bootstrap required, opened a new log
I20260811 03:00:48.857761 8189 ts_tablet_manager.cc:1397] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 03:00:48.858131 8189 raft_consensus.cc:348] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.858224 8189 raft_consensus.cc:374] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:48.858260 8189 raft_consensus.cc:729] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b51efa70c449429918a62cc6d934c9, State: Initialized, Role: FOLLOWER
I20260811 03:00:48.858388 8189 consensus_queue.cc:260] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.858639 8189 ts_tablet_manager.cc:1428] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 03:00:48.859340 8519 tablet_service.cc:1467] Processing CreateTablet for tablet 8fabb6959ccc4b0baaa582a9ba2f1496 (DEFAULT_TABLE table=test-table [id=2220e10b387b4332a3866e7fe9d21bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:48.860388 8519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fabb6959ccc4b0baaa582a9ba2f1496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:48.860183 8774 tablet_service.cc:1467] Processing CreateTablet for tablet 8fabb6959ccc4b0baaa582a9ba2f1496 (DEFAULT_TABLE table=test-table [id=2220e10b387b4332a3866e7fe9d21bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:00:48.861066 8774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fabb6959ccc4b0baaa582a9ba2f1496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:00:48.866775 8932 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Bootstrap starting.
I20260811 03:00:48.866941 8934 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Bootstrap starting.
I20260811 03:00:48.868817 8934 tablet_bootstrap.cc:654] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:48.869349 8932 tablet_bootstrap.cc:654] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Neither blocks nor log segments found. Creating new log.
I20260811 03:00:48.869858 8934 log.cc:824] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:48.870010 8932 log.cc:824] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Log is configured to *not* fsync() on all Append() calls
I20260811 03:00:48.871387 8934 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: No bootstrap required, opened a new log
I20260811 03:00:48.871547 8934 ts_tablet_manager.cc:1397] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 03:00:48.871556 8932 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: No bootstrap required, opened a new log
I20260811 03:00:48.871708 8932 ts_tablet_manager.cc:1397] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:00:48.874441 8934 raft_consensus.cc:348] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.874648 8934 raft_consensus.cc:374] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:48.874552 8932 raft_consensus.cc:348] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.874703 8934 raft_consensus.cc:729] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 692e9d80bb0c4c3085a9773ced2a2076, State: Initialized, Role: FOLLOWER
I20260811 03:00:48.874768 8932 raft_consensus.cc:374] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:00:48.874821 8932 raft_consensus.cc:729] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd070f7db2943c2aed5b65f91526348, State: Initialized, Role: FOLLOWER
I20260811 03:00:48.875097 8934 consensus_queue.cc:260] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.875183 8932 consensus_queue.cc:260] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:48.875622 8904 heartbeater.cc:503] Master 127.8.203.126:41919 was elected leader, sending a full tablet report...
I20260811 03:00:48.875752 8650 heartbeater.cc:503] Master 127.8.203.126:41919 was elected leader, sending a full tablet report...
I20260811 03:00:48.876818 8934 ts_tablet_manager.cc:1428] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:00:48.876842 8932 ts_tablet_manager.cc:1428] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 03:00:48.938058 8385 tablet.cc:2369] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:00:49.057089 8906 tablet.cc:2369] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:00:49.119669 8651 tablet.cc:2369] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:00:49.181058 8939 raft_consensus.cc:482] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:00:49.181342 8939 raft_consensus.cc:504] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:49.182790 8939 leader_election.cc:288] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373), 34b51efa70c449429918a62cc6d934c9 (127.8.203.65:45401)
I20260811 03:00:49.188233 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:49.183133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:35111 (local address 127.8.203.66:38373)
0811 03:00:49.183274 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:49.183280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:49.183741 (+ 461us) server_negotiation.cc:407] Connection header received
0811 03:00:49.183880 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:49.183883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:49.183974 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:49.184140 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:49.184585 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.185324 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.186837 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.187319 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.187418 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:49.187474 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:49.188008 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 03:00:49.188090 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 03:00:49.188292 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:49.183024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45401 (local address 127.8.203.67:56389)
0811 03:00:49.183204 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:49.183220 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:49.183321 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:49.184036 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:49.184052 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:49.184071 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:49.184196 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:49.184203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.185262 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.185266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:49.186636 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:49.186649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.187822 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.187826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:49.187970 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:49.188000 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 03:00:49.188058 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 03:00:49.188135 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":70}
I20260811 03:00:49.188943 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:49.183024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38373 (local address 127.8.203.67:35111)
0811 03:00:49.183645 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:49.183660 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:00:49.183770 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:00:49.184285 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:00:49.184288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:00:49.184310 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:00:49.184431 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:49.184438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.185466 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.185469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:49.186685 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:00:49.186698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.187764 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.187767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:00:49.187862 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:49.187886 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:00:49.188741 (+ 855us) client_negotiation.cc:240] Negotiation successful
0811 03:00:49.188811 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":246,"thread_start_us":136,"threads_started":1}
I20260811 03:00:49.188984 8943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:49.183353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56389 (local address 127.8.203.65:45401)
0811 03:00:49.183643 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:49.183648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:49.183666 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:00:49.183801 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:49.183804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:49.183884 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:49.184026 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:00:49.184315 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.185138 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.186855 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.187666 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.188504 (+ 838us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:49.188698 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:49.188742 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 03:00:49.188826 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260811 03:00:49.192580 8559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fabb6959ccc4b0baaa582a9ba2f1496" candidate_uuid: "692e9d80bb0c4c3085a9773ced2a2076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd070f7db2943c2aed5b65f91526348" is_pre_election: true
I20260811 03:00:49.192569 8300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fabb6959ccc4b0baaa582a9ba2f1496" candidate_uuid: "692e9d80bb0c4c3085a9773ced2a2076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b51efa70c449429918a62cc6d934c9" is_pre_election: true
I20260811 03:00:49.192865 8300 raft_consensus.cc:2454] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 692e9d80bb0c4c3085a9773ced2a2076 in term 0.
I20260811 03:00:49.192885 8559 raft_consensus.cc:2454] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 692e9d80bb0c4c3085a9773ced2a2076 in term 0.
I20260811 03:00:49.193384 8682 leader_election.cc:302] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9, 692e9d80bb0c4c3085a9773ced2a2076, 7fd070f7db2943c2aed5b65f91526348; no voters:
I20260811 03:00:49.193650 8939 raft_consensus.cc:2781] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:00:49.193732 8939 raft_consensus.cc:482] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:00:49.193778 8939 raft_consensus.cc:3037] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:49.194911 8939 raft_consensus.cc:504] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:49.195257 8939 leader_election.cc:288] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [CANDIDATE]: Term 1 election: Requested vote from peers 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373), 34b51efa70c449429918a62cc6d934c9 (127.8.203.65:45401)
I20260811 03:00:49.195612 8300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fabb6959ccc4b0baaa582a9ba2f1496" candidate_uuid: "692e9d80bb0c4c3085a9773ced2a2076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b51efa70c449429918a62cc6d934c9"
I20260811 03:00:49.195627 8559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fabb6959ccc4b0baaa582a9ba2f1496" candidate_uuid: "692e9d80bb0c4c3085a9773ced2a2076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fd070f7db2943c2aed5b65f91526348"
I20260811 03:00:49.195762 8300 raft_consensus.cc:3037] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:49.195762 8559 raft_consensus.cc:3037] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:00:49.197047 8559 raft_consensus.cc:2454] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 692e9d80bb0c4c3085a9773ced2a2076 in term 1.
I20260811 03:00:49.197047 8300 raft_consensus.cc:2454] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 692e9d80bb0c4c3085a9773ced2a2076 in term 1.
I20260811 03:00:49.197387 8682 leader_election.cc:302] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9, 692e9d80bb0c4c3085a9773ced2a2076; no voters:
I20260811 03:00:49.197589 8939 raft_consensus.cc:2781] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:00:49.197783 8939 raft_consensus.cc:686] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 LEADER]: Becoming Leader. State: Replica: 692e9d80bb0c4c3085a9773ced2a2076, State: Running, Role: LEADER
I20260811 03:00:49.198088 8939 consensus_queue.cc:237] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:00:49.201541 8030 catalog_manager.cc:5495] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 reported cstate change: term changed from 0 to 1, leader changed from <none> to 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67). New cstate: current_term: 1 leader_uuid: "692e9d80bb0c4c3085a9773ced2a2076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } health_report { overall_health: UNKNOWN } } }
I20260811 03:00:49.220160 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:00:49.216823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.8.203.67:43947)
0811 03:00:49.216937 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 03:00:49.216942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:00:49.216956 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:00:49.217116 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:00:49.217118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:00:49.217168 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:00:49.217255 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:00:49.217515 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.218003 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.218810 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:00:49.219025 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:00:49.219110 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:00:49.219399 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:00:49.219783 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:00:49.219893 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 03:00:49.219966 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 03:00:49.220017 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:00:49.225737 8300 raft_consensus.cc:1261] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Refusing update from remote peer 692e9d80bb0c4c3085a9773ced2a2076: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:49.225741 8559 raft_consensus.cc:1261] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Refusing update from remote peer 692e9d80bb0c4c3085a9773ced2a2076: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:00:49.226392 8944 consensus_queue.cc:1035] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:49.226588 8939 consensus_queue.cc:1035] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:00:49.239884 8030 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:49.240024 8030 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:00:49.242079 8030 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Sending DeleteTablet for 3 replicas of tablet 8fabb6959ccc4b0baaa582a9ba2f1496
I20260811 03:00:49.242733 8268 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fabb6959ccc4b0baaa582a9ba2f1496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:49 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:59504
I20260811 03:00:49.242779 8774 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fabb6959ccc4b0baaa582a9ba2f1496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:49 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:48038
I20260811 03:00:49.243223 8978 tablet_replica.cc:330] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: stopping tablet replica
I20260811 03:00:49.243361 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 8107
I20260811 03:00:49.243597 8978 raft_consensus.cc:2229] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:00:49.248769 8978 raft_consensus.cc:2258] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:49.251762 8978 ts_tablet_manager.cc:1905] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:49.255297 8978 ts_tablet_manager.cc:1918] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:49.255494 8978 log.cc:1196] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/8fabb6959ccc4b0baaa582a9ba2f1496
I20260811 03:00:49.256095 8978 ts_tablet_manager.cc:1939] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 692e9d80bb0c4c3085a9773ced2a2076: Deleting consensus metadata
I20260811 03:00:49.257727 8010 catalog_manager.cc:4841] TS 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947): tablet 8fabb6959ccc4b0baaa582a9ba2f1496 (table test-table [id=2220e10b387b4332a3866e7fe9d21bca]) successfully deleted
I20260811 03:00:49.264019 8519 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fabb6959ccc4b0baaa582a9ba2f1496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:00:49 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:33460
I20260811 03:00:49.264611 8979 tablet_replica.cc:330] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: stopping tablet replica
I20260811 03:00:49.264952 8979 raft_consensus.cc:2229] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:00:49.265312 8979 raft_consensus.cc:2258] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:00:49.266865 8979 ts_tablet_manager.cc:1905] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:00:49.269502 8979 ts_tablet_manager.cc:1918] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:00:49.269598 8979 log.cc:1196] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/8fabb6959ccc4b0baaa582a9ba2f1496
I20260811 03:00:49.269888 8979 ts_tablet_manager.cc:1939] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 7fd070f7db2943c2aed5b65f91526348: Deleting consensus metadata
W20260811 03:00:49.271140 8007 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:45401 (error 108)
I20260811 03:00:49.271356 8010 catalog_manager.cc:4841] TS 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373): tablet 8fabb6959ccc4b0baaa582a9ba2f1496 (table test-table [id=2220e10b387b4332a3866e7fe9d21bca]) successfully deleted
W20260811 03:00:49.272187 8399 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:00:49.271935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:45401 (local address 127.0.0.1:59516)
0811 03:00:49.272057 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 03:00:49.272118 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45401: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 03:00:49.272334 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 8389
W20260811 03:00:49.284538 8007 catalog_manager.cc:4584] TS 34b51efa70c449429918a62cc6d934c9 (127.8.203.65:45401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8fabb6959ccc4b0baaa582a9ba2f1496: Network error: Client connection negotiation failed: client connection to 127.8.203.65:45401: connect: Connection reset by peer (error 104)
I20260811 03:00:49.289783 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 8654
I20260811 03:00:49.306475 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 7972
I20260811 03:01:09.319876 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41919
--webserver_interface=127.8.203.126
--webserver_port=38175
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:41919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:09.430141 8983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:09.430401 8983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:09.430457 8983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:09.434388 8983 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 03:01:09.434438 8983 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 03:01:09.434459 8983 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 03:01:09.434476 8983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:09.434500 8983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:09.434533 8983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:09.434557 8983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:09.434590 8983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:09.439689 8983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:41919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=38175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:09.440659 8983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:09.441670 8983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:09.450033 8998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:09.450069 8991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:09.450212 8983 server_base.cc:1034] running on GCE node
W20260811 03:01:09.450099 8995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:09.450773 8983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:09.451763 8983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:09.452907 8983 hybrid_clock.cc:648] HybridClock initialized: now 1786417269452885 us; error 40 us; skew 500 ppm
Aug 11 03:01:09 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:09.455453 8983 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:01:09.456319 8983 webserver.cc:460] Webserver started at http://127.8.203.126:38175/ using document root <none> and password file <none>
I20260811 03:01:09.456835 8983 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:09.456898 8983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:09.460608 8983 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:09.462643 9027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:09.463792 8983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:09.463910 8983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "d1829f20ad1e467daf847875f99a238a"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "d5308f5aaa5ec21226af834f297826a3"
server_key_iv: "cbfeb25ec2bb2d3ac61cae667aea17b5"
server_key_version: "encryptionkey@0"
I20260811 03:01:09.464395 8983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:09.489192 8983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:09.491876 8983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:09.492046 8983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:09.498895 8983 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:41919
I20260811 03:01:09.499158 9126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:41919 every 8 connection(s)
I20260811 03:01:09.499936 8983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:09.504122 9127 sys_catalog.cc:263] Verifying existing consensus state
I20260811 03:01:09.505759 9127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Bootstrap starting.
I20260811 03:01:09.506588 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 8983
I20260811 03:01:09.507066 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:45401
--local_ip_for_outbound_sockets=127.8.203.65
--tserver_master_addrs=127.8.203.126:41919
--webserver_port=33109
--webserver_interface=127.8.203.65
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:01:09.523329 9127 log.cc:824] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:09.534571 9127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:01:09.534901 9127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Bootstrap complete.
I20260811 03:01:09.538744 9127 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:01:09.539120 9127 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1829f20ad1e467daf847875f99a238a, State: Initialized, Role: FOLLOWER
I20260811 03:01:09.539585 9127 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:01:09.539700 9127 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:09.539762 9127 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:09.539853 9127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 2 FOLLOWER]: Advancing to term 3
I20260811 03:01:09.541646 9127 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:01:09.541949 9127 leader_election.cc:302] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1829f20ad1e467daf847875f99a238a; no voters:
I20260811 03:01:09.542250 9127 leader_election.cc:288] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 03:01:09.542346 9137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 3 FOLLOWER]: Leader election won for term 3
I20260811 03:01:09.542572 9137 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [term 3 LEADER]: Becoming Leader. State: Replica: d1829f20ad1e467daf847875f99a238a, State: Running, Role: LEADER
I20260811 03:01:09.542932 9137 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } }
I20260811 03:01:09.543442 9127 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:09.543819 9137 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1829f20ad1e467daf847875f99a238a. Latest consensus state: current_term: 3 leader_uuid: "d1829f20ad1e467daf847875f99a238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } } }
I20260811 03:01:09.543937 9137 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:09.544294 9144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:09.544299 9138 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d1829f20ad1e467daf847875f99a238a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1829f20ad1e467daf847875f99a238a" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41919 } } }
I20260811 03:01:09.544566 9138 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:09.549163 9144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:09.549821 9144 catalog_manager.cc:1259] Loaded cluster ID: a4d008e7c9a94f238c185328caa038af
I20260811 03:01:09.549880 9144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:09.551581 9144 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:09.552213 9144 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Loaded TSK: 0
I20260811 03:01:09.552287 9144 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d1829f20ad1e467daf847875f99a238a: Loaded TSK: 1
F20260811 03:01:09.552958 9144 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe0288579ad google::LogMessage::Fail() at ??:0
@ 0x7fe02885a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe0288574de google::LogMessage::Flush() at ??:0
@ 0x7fe028858a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe02b51b18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe02b513f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe0331bc7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe03319699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe033197178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fe0331d8df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe0331f782b std::function<>::operator()() at ??:0
@ 0x7fe033196b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe033197f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe033194579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe0331d8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe02aaa93d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe02aaa9c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe02aaab3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe02aa9a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe02c0de6db start_thread at ??:0
@ 0x7fe02735561f clone at ??:0
W20260811 03:01:09.646996 9132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:09.647316 9132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:09.647393 9132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:09.653259 9132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:09.653347 9132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:09.653443 9132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:01:09.661043 9132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:45401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=33109
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:09.662401 9132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:09.663704 9132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:09.674547 9216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:09.674654 9132 server_base.cc:1034] running on GCE node
W20260811 03:01:09.674571 9213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:09.674791 9214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:09.675318 9132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:09.676116 9132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:09.677336 9132 hybrid_clock.cc:648] HybridClock initialized: now 1786417269677317 us; error 25 us; skew 500 ppm
Aug 11 03:01:09 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:09.680467 9132 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:01:09.681267 9132 webserver.cc:460] Webserver started at http://127.8.203.65:33109/ using document root <none> and password file <none>
I20260811 03:01:09.681828 9132 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:09.681888 9132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:09.685768 9132 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:01:09.687955 9228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:09.689183 9132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:09.689324 9132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "34b51efa70c449429918a62cc6d934c9"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "ecfae09fe54afe55f234f0e04d59b8e7"
server_key_iv: "ebda4d13fe01bed6edd4194daef55c42"
server_key_version: "encryptionkey@0"
I20260811 03:01:09.689872 9132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:09.742857 9132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:09.746161 9132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:09.746343 9132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:09.746950 9132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:09.748046 9245 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 03:01:09.753656 9132 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 03:01:09.753722 9132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 03:01:09.753795 9132 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 03:01:09.755550 9132 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 03:01:09.755597 9132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 03:01:09.755946 9245 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Bootstrap starting.
I20260811 03:01:09.765462 9132 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:45401
I20260811 03:01:09.765627 9451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:45401 every 8 connection(s)
I20260811 03:01:09.766556 9132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:01:09.775314 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 9132
I20260811 03:01:09.775738 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:38373
--local_ip_for_outbound_sockets=127.8.203.66
--tserver_master_addrs=127.8.203.126:41919
--webserver_port=35417
--webserver_interface=127.8.203.66
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:01:09.777264 9245 log.cc:824] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:09.793700 9245 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 03:01:09.794045 9245 tablet_bootstrap.cc:492] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Bootstrap complete.
W20260811 03:01:09.794325 9458 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:01:09.768396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:60529)
0811 03:01:09.768879 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.768935 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:09.769766 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:09.793194 (+ 23428us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
W20260811 03:01:09.794898 9454 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:41919 (0 consecutive failures): Network error: Failed to ping master at 127.8.203.126:41919: Client connection negotiation failed: client connection to 127.8.203.126:41919: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 03:01:09.795219 9245 ts_tablet_manager.cc:1397] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Time spent bootstrapping tablet: real 0.040s user 0.019s sys 0.011s
I20260811 03:01:09.795668 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:09.795404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:56077)
0811 03:01:09.795538 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.795607 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 03:01:09.796382 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:09.796137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:56033)
0811 03:01:09.796281 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.796326 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 03:01:09.796623 9454 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 03:01:09.797991 9245 raft_consensus.cc:348] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:01:09.798400 9245 raft_consensus.cc:729] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b51efa70c449429918a62cc6d934c9, State: Initialized, Role: FOLLOWER
I20260811 03:01:09.798861 9245 consensus_queue.cc:260] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:01:09.799849 9245 ts_tablet_manager.cc:1428] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 03:01:09.799889 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:09.799634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:52805)
0811 03:01:09.799783 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.799830 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 03:01:09.800452 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:09.800260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:48533)
0811 03:01:09.800363 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.800408 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 03:01:09.896721 9459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:09.896956 9459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:09.897013 9459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:09.900882 9459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:09.900933 9459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:09.901007 9459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:01:09.906060 9459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:38373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=35417
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:09.907014 9459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:09.907974 9459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:09.916999 9466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:09.917029 9467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:09.917040 9470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:09.917328 9459 server_base.cc:1034] running on GCE node
I20260811 03:01:09.917770 9459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:09.918347 9459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:09.919508 9459 hybrid_clock.cc:648] HybridClock initialized: now 1786417269919484 us; error 40 us; skew 500 ppm
Aug 11 03:01:09 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:09.922025 9459 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:01:09.922816 9459 webserver.cc:460] Webserver started at http://127.8.203.66:35417/ using document root <none> and password file <none>
I20260811 03:01:09.923298 9459 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:09.923354 9459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:09.926941 9459 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:01:09.928831 9485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:09.930065 9459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:09.930213 9459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "7fd070f7db2943c2aed5b65f91526348"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "9318ea34def1123d8fd4d891725e18d4"
server_key_iv: "b5672844aa8b3528356b95300282351f"
server_key_version: "encryptionkey@0"
I20260811 03:01:09.931020 9459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:09.946558 9459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:09.949631 9459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:09.949800 9459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:09.950361 9459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:09.951176 9459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:09.951223 9459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:09.951267 9459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:09.951295 9459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:09.959746 9459 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:38373
I20260811 03:01:09.959780 9698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:38373 every 8 connection(s)
I20260811 03:01:09.960687 9459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:01:09.964370 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:09.962672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:36621)
0811 03:01:09.963109 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.963210 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":117,"threads_started":1}
W20260811 03:01:09.964797 9699 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:41919 (0 consecutive failures): Network error: Failed to ping master at 127.8.203.126:41919: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
I20260811 03:01:09.965328 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:09.965109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:36819)
0811 03:01:09.965207 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.965255 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 03:01:09.965803 9704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:09.965638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:53993)
0811 03:01:09.965714 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:09.965760 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 03:01:09.965934 9699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 03:01:09.969827 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 9459
I20260811 03:01:09.970206 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:43947
--local_ip_for_outbound_sockets=127.8.203.67
--tserver_master_addrs=127.8.203.126:41919
--webserver_port=37565
--webserver_interface=127.8.203.67
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:10.080686 9705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:10.080921 9705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:10.080966 9705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:10.084885 9705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:10.084939 9705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:10.085008 9705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:01:10.090111 9705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:43947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=37565
--tserver_master_addrs=127.8.203.126:41919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:10.091068 9705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:10.092015 9705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:10.101380 9713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:10.101375 9717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:10.101511 9705 server_base.cc:1034] running on GCE node
W20260811 03:01:10.101682 9712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:10.102046 9705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:10.102625 9705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:10.103777 9705 hybrid_clock.cc:648] HybridClock initialized: now 1786417270103750 us; error 42 us; skew 500 ppm
Aug 11 03:01:10 dist-test-slave-gjw7 krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417270, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:10.106360 9705 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:01:10.107172 9705 webserver.cc:460] Webserver started at http://127.8.203.67:37565/ using document root <none> and password file <none>
I20260811 03:01:10.107692 9705 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:10.107757 9705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:10.111402 9705 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:01:10.113449 9729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:10.114627 9705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:10.114741 9705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "692e9d80bb0c4c3085a9773ced2a2076"
format_stamp: "Formatted at 2026-08-11 03:00:27 on dist-test-slave-gjw7"
server_key: "b0587ec45ac6198e73d7a1053a0cd1fa"
server_key_iv: "d6c911b3c57ceeda0264249556fafea4"
server_key_version: "encryptionkey@0"
I20260811 03:01:10.115260 9705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:10.136732 9705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:10.139474 9705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:10.139622 9705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:10.140141 9705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:10.140928 9705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:10.140973 9705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:10.141019 9705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:10.141040 9705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:10.149471 9705 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:43947
I20260811 03:01:10.149507 9975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:43947 every 8 connection(s)
I20260811 03:01:10.150432 9705 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:01:10.154117 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:10.152416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:54151)
0811 03:01:10.152844 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:10.152954 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":115,"threads_started":1}
W20260811 03:01:10.154526 9976 heartbeater.cc:644] Failed to heartbeat to 127.8.203.126:41919 (0 consecutive failures): Network error: Failed to ping master at 127.8.203.126:41919: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
I20260811 03:01:10.155066 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:10.154877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:32873)
0811 03:01:10.154967 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:10.155013 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 03:01:10.155593 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:10.155439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:42979)
0811 03:01:10.155512 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:10.155550 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 03:01:10.155764 9976 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 03:01:10.157342 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 9705
I20260811 03:01:10.801512 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:10.801056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:44947)
0811 03:01:10.801346 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:10.801410 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":75,"threads_started":1}
I20260811 03:01:10.966781 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:10.966403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:46603)
0811 03:01:10.966631 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:10.966682 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260811 03:01:11.156595 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:11.156229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:53999)
0811 03:01:11.156448 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:11.156499 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":70,"threads_started":1}
I20260811 03:01:11.507897 9988 raft_consensus.cc:482] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:11.508095 9988 raft_consensus.cc:504] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:01:11.509192 9988 leader_election.cc:288] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947), 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
Aug 11 03:01:11 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.66@KRBTEST.COM
Aug 11 03:01:11 dist-test-slave-gjw7 krb5kdc[6874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.67@KRBTEST.COM
I20260811 03:01:11.522285 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:11.509744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38373 (local address 127.8.203.65:50985)
0811 03:01:11.510026 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:11.510038 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:11.510141 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:11.511543 (+ 1402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:11.511549 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:11.512073 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:11.512535 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:11.512549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.513928 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.513932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:11.514806 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:11.514816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.515467 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.515470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:11.515638 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:11.516233 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:11.516255 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:11.517869 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:11.519928 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:11.519934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:11.519949 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:11.520259 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:11.520570 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:11.520573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:11.520575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:11.520743 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:11.521406 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:11.521411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:11.521653 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 03:01:11.521853 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 03:01:11.522134 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":71,"spinlock_wait_cycles":13440,"thread_start_us":48,"threads_started":1}
I20260811 03:01:11.522392 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:11.509916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:50985 (local address 127.8.203.66:38373)
0811 03:01:11.510246 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:11.510258 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:11.510293 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:01:11.510926 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:11.510931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:11.511215 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:11.511467 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:11.512692 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.513792 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.514921 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.515357 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.515664 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:11.518087 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:11.518092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:11.518102 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:11.518129 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:11.519806 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:11.520361 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:11.520366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:11.520370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:11.520481 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:11.520825 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:11.520828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:11.520830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:11.521309 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:11.521439 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:11.521958 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 03:01:11.522213 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
I20260811 03:01:11.522575 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:11.509397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:43947 (local address 127.8.203.65:41247)
0811 03:01:11.509619 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:11.509631 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:11.509736 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:11.511544 (+ 1808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:11.511549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:11.512072 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:11.512534 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:11.512549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.513957 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.513961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:11.514805 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:11.514815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.515723 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.515727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:11.515815 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:11.516387 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:11.516403 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:11.518645 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:11.520777 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:11.520779 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:11.520781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:11.521014 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:11.521325 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:11.521328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:11.521329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:11.521433 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:11.522087 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:11.522091 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:11.522179 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 03:01:11.522284 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 03:01:11.522423 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":57,"spinlock_wait_cycles":896,"thread_start_us":62,"threads_started":1}
I20260811 03:01:11.522864 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:11.509603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:41247 (local address 127.8.203.67:43947)
0811 03:01:11.510062 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:11.510089 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:11.510138 (+ 49us) server_negotiation.cc:407] Connection header received
0811 03:01:11.510828 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:11.510844 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:11.511187 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:11.511467 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:11.512693 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.513827 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.514920 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:11.515382 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:11.515538 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:11.518818 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:11.518823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:11.518834 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:11.518860 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:11.520674 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:11.521110 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:11.521114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:11.521118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:11.521218 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:11.521519 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:11.521522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:11.521524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:11.521979 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:11.522098 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:11.522439 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 03:01:11.522698 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":110,"threads_started":1}
W20260811 03:01:11.525031 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
W20260811 03:01:11.525205 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
I20260811 03:01:11.525264 9233 leader_election.cc:302] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9; no voters: 692e9d80bb0c4c3085a9773ced2a2076, 7fd070f7db2943c2aed5b65f91526348
I20260811 03:01:11.525508 9988 raft_consensus.cc:2726] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:01:11.802397 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:11.802137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:46611)
0811 03:01:11.802254 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:11.802306 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 03:01:11.967789 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:11.967384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:51593)
0811 03:01:11.967629 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:11.967683 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260811 03:01:12.157771 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:12.157276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:53189)
0811 03:01:12.157588 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:12.157653 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":86,"threads_started":1}
I20260811 03:01:12.803539 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:12.803079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:43403)
0811 03:01:12.803386 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:12.803441 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260811 03:01:12.968788 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:12.968416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:56421)
0811 03:01:12.968647 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:12.968699 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
I20260811 03:01:13.158843 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:13.158444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:39697)
0811 03:01:13.158707 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:13.158758 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":89,"threads_started":1}
I20260811 03:01:13.455268 10007 raft_consensus.cc:482] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:13.455406 10007 raft_consensus.cc:504] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:01:13.455751 10007 leader_election.cc:288] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947), 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
W20260811 03:01:13.456588 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
W20260811 03:01:13.456740 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
I20260811 03:01:13.456786 9233 leader_election.cc:302] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9; no voters: 692e9d80bb0c4c3085a9773ced2a2076, 7fd070f7db2943c2aed5b65f91526348
I20260811 03:01:13.456954 10007 raft_consensus.cc:2726] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:01:13.804571 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:13.804184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:36431)
0811 03:01:13.804422 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:13.804477 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260811 03:01:13.969856 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:13.969471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:43993)
0811 03:01:13.969699 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:13.969752 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
I20260811 03:01:14.159874 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:14.159482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:49783)
0811 03:01:14.159718 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:14.159771 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260811 03:01:14.805791 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:14.805411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:51235)
0811 03:01:14.805644 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:14.805695 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260811 03:01:14.970919 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:14.970521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:44201)
0811 03:01:14.970772 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:14.970826 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
I20260811 03:01:15.160946 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:15.160569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:52683)
0811 03:01:15.160792 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:15.160853 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260811 03:01:15.806847 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:15.806468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:41989)
0811 03:01:15.806697 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:15.806748 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":79,"threads_started":1}
I20260811 03:01:15.864778 10026 raft_consensus.cc:482] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:15.864921 10026 raft_consensus.cc:504] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:01:15.865337 10026 leader_election.cc:288] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947), 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
W20260811 03:01:15.866005 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
W20260811 03:01:15.866127 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
I20260811 03:01:15.866163 9233 leader_election.cc:302] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9; no voters: 692e9d80bb0c4c3085a9773ced2a2076, 7fd070f7db2943c2aed5b65f91526348
I20260811 03:01:15.866286 10026 raft_consensus.cc:2726] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:01:15.971952 10027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:15.971531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:43957)
0811 03:01:15.971800 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:15.971861 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":98,"threads_started":1}
I20260811 03:01:16.161947 10028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:16.161557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:52909)
0811 03:01:16.161797 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:16.161859 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":86,"threads_started":1}
I20260811 03:01:16.807915 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:16.807513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:34621)
0811 03:01:16.807756 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:16.807808 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260811 03:01:16.973071 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:16.972602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:54989)
0811 03:01:16.972919 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:16.972974 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":89,"threads_started":1}
I20260811 03:01:17.162982 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:17.162542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:32969)
0811 03:01:17.162843 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:17.162895 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":88,"threads_started":1}
I20260811 03:01:17.809031 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:17.808615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:58639)
0811 03:01:17.808876 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:17.808933 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260811 03:01:17.974062 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:17.973668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:60085)
0811 03:01:17.973917 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:17.973970 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260811 03:01:18.164052 10037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:18.163637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:39127)
0811 03:01:18.163891 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:18.163950 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260811 03:01:18.810173 10038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:18.809733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:55863)
0811 03:01:18.810019 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:18.810075 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":106,"threads_started":1}
I20260811 03:01:18.975083 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:18.974704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:39195)
0811 03:01:18.974936 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:18.974988 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260811 03:01:19.165050 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:19.164666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:39151)
0811 03:01:19.164908 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:19.164959 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260811 03:01:19.811255 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:19.810875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:34623)
0811 03:01:19.811107 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:19.811158 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":72,"threads_started":1}
I20260811 03:01:19.976114 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:19.975722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:49353)
0811 03:01:19.975975 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:19.976023 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260811 03:01:20.166220 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:20.165796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:47395)
0811 03:01:20.166069 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:20.166122 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":106,"threads_started":1}
I20260811 03:01:20.702528 10055 raft_consensus.cc:482] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:20.702656 10055 raft_consensus.cc:504] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "692e9d80bb0c4c3085a9773ced2a2076" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 43947 } } peers { permanent_uuid: "7fd070f7db2943c2aed5b65f91526348" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38373 } } peers { permanent_uuid: "34b51efa70c449429918a62cc6d934c9" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 45401 } }
I20260811 03:01:20.703058 10055 leader_election.cc:288] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947), 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373)
W20260811 03:01:20.703758 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 692e9d80bb0c4c3085a9773ced2a2076 (127.8.203.67:43947): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
W20260811 03:01:20.703914 9233 leader_election.cc:341] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fd070f7db2943c2aed5b65f91526348 (127.8.203.66:38373): Not found: Tablet not found: 8fabb6959ccc4b0baaa582a9ba2f1496
I20260811 03:01:20.703969 9233 leader_election.cc:302] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34b51efa70c449429918a62cc6d934c9; no voters: 692e9d80bb0c4c3085a9773ced2a2076, 7fd070f7db2943c2aed5b65f91526348
I20260811 03:01:20.704126 10055 raft_consensus.cc:2726] T 8fabb6959ccc4b0baaa582a9ba2f1496 P 34b51efa70c449429918a62cc6d934c9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 03:01:20.812314 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:20.811921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:42891)
0811 03:01:20.812165 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:20.812218 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260811 03:01:20.977308 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:20.976825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:56761)
0811 03:01:20.977121 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:20.977183 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":112,"threads_started":1}
I20260811 03:01:21.167240 10059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:21.166851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:47981)
0811 03:01:21.167102 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:21.167152 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
I20260811 03:01:21.813529 10060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:21.812991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:44053)
0811 03:01:21.813314 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:21.813419 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":106,"threads_started":1}
I20260811 03:01:21.978427 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:21.978002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:39051)
0811 03:01:21.978265 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:21.978320 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":93,"threads_started":1}
I20260811 03:01:22.168324 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:22.167901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:59987)
0811 03:01:22.168172 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:22.168227 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260811 03:01:22.814594 10063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:22.814183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:42575)
0811 03:01:22.814437 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:22.814494 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
I20260811 03:01:22.979511 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:22.979094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:53029)
0811 03:01:22.979356 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:22.979413 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
I20260811 03:01:23.169488 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:23.169000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:59007)
0811 03:01:23.169272 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:23.169390 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260811 03:01:23.815646 10068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:23.815241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:38215)
0811 03:01:23.815497 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:23.815550 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260811 03:01:23.980659 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:23.980224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:41159)
0811 03:01:23.980508 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:23.980562 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":103,"threads_started":1}
I20260811 03:01:24.170539 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:24.170126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:50177)
0811 03:01:24.170391 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:24.170445 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":91,"threads_started":1}
I20260811 03:01:24.816856 10076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:24.816448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.65:52059)
0811 03:01:24.816697 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:24.816747 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260811 03:01:24.981932 10077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:24.981493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.66:55341)
0811 03:01:24.981764 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:24.981822 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":89,"threads_started":1}
I20260811 03:01:25.165951 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 9132
I20260811 03:01:25.175661 10080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:25.175247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41919 (local address 127.8.203.67:54697)
0811 03:01:25.175515 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:25.175569 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.126:41919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":70,"threads_started":1}
I20260811 03:01:25.181376 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 9459
I20260811 03:01:25.189998 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 9705
2026-08-11T03:01:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59555 ms)
[----------] 30 tests from SecurityITest (246170 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:25 dist-test-slave-gjw7 krb5kdc[10084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:25 dist-test-slave-gjw7 krb5kdc[10084](info): set up 2 sockets
Aug 11 03:01:25 dist-test-slave-gjw7 krb5kdc[10084](info): commencing operation
krb5kdc: starting...
W20260811 03:01:26.397097 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:26 dist-test-slave-gjw7 krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:01:26.512238 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:33941
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:33941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:26.623982 10115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:26.624260 10115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:26.624301 10115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:26.628170 10115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:26.628224 10115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:26.628245 10115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:26.628263 10115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:26.628280 10115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:26.633343 10115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:33941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:33941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:26.634277 10115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:26.635233 10115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:26.643498 10126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:26.643543 10131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:26.643497 10124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:26.643728 10115 server_base.cc:1034] running on GCE node
I20260811 03:01:26.644266 10115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:26.645339 10115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:26.646502 10115 hybrid_clock.cc:648] HybridClock initialized: now 1786417286646489 us; error 24 us; skew 500 ppm
Aug 11 03:01:26 dist-test-slave-gjw7 krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:26.649070 10115 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:01:26.649912 10115 webserver.cc:460] Webserver started at http://127.8.203.126:43361/ using document root <none> and password file <none>
I20260811 03:01:26.650434 10115 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:26.650498 10115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:26.650694 10115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:26.652630 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "01685436cd444e3fa9bbdc8218f8c84e"
format_stamp: "Formatted at 2026-08-11 03:01:26 on dist-test-slave-gjw7"
server_key: "5e94bbd5aaf4e8f4f8031e458c5a1b1a"
server_key_iv: "1f9724b6adacdfb5bc9004c490e52ff5"
server_key_version: "encryptionkey@0"
I20260811 03:01:26.653154 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "01685436cd444e3fa9bbdc8218f8c84e"
format_stamp: "Formatted at 2026-08-11 03:01:26 on dist-test-slave-gjw7"
server_key: "5e94bbd5aaf4e8f4f8031e458c5a1b1a"
server_key_iv: "1f9724b6adacdfb5bc9004c490e52ff5"
server_key_version: "encryptionkey@0"
I20260811 03:01:26.656723 10115 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:01:26.658970 10145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:26.660102 10115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 03:01:26.660210 10115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "01685436cd444e3fa9bbdc8218f8c84e"
format_stamp: "Formatted at 2026-08-11 03:01:26 on dist-test-slave-gjw7"
server_key: "5e94bbd5aaf4e8f4f8031e458c5a1b1a"
server_key_iv: "1f9724b6adacdfb5bc9004c490e52ff5"
server_key_version: "encryptionkey@0"
I20260811 03:01:26.660297 10115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:26.681772 10115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:26.684326 10115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:26.684499 10115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:26.691020 10115 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:33941
I20260811 03:01:26.691038 10256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:33941 every 8 connection(s)
I20260811 03:01:26.691901 10115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:26.694587 10257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:26.697415 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10115
I20260811 03:01:26.697528 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:26.697772 9005 external_mini_cluster.cc:1428] Setting key 74be91ff80dec2ded229346fa6703130
I20260811 03:01:26.700380 10257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e: Bootstrap starting.
I20260811 03:01:26.702500 10257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:26.703200 10257 log.cc:824] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:26.705031 10257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e: No bootstrap required, opened a new log
Aug 11 03:01:26 dist-test-slave-gjw7 krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:26.708765 10257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01685436cd444e3fa9bbdc8218f8c84e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33941 } }
I20260811 03:01:26.709039 10257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:26.709115 10257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01685436cd444e3fa9bbdc8218f8c84e, State: Initialized, Role: FOLLOWER
I20260811 03:01:26.709596 10257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01685436cd444e3fa9bbdc8218f8c84e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33941 } }
I20260811 03:01:26.709749 10257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:26.709832 10257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:26.709939 10257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:26.710937 10257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01685436cd444e3fa9bbdc8218f8c84e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33941 } }
I20260811 03:01:26.711280 10257 leader_election.cc:302] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01685436cd444e3fa9bbdc8218f8c84e; no voters:
I20260811 03:01:26.711601 10257 leader_election.cc:288] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:26.711707 10267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:26.711896 10267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [term 1 LEADER]: Becoming Leader. State: Replica: 01685436cd444e3fa9bbdc8218f8c84e, State: Running, Role: LEADER
I20260811 03:01:26.711918 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:26.699062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.8.203.126:33941)
0811 03:01:26.699492 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:26.699507 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:26.699541 (+ 34us) server_negotiation.cc:407] Connection header received
0811 03:01:26.700137 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:26.700154 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:26.700458 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:26.700778 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:26.701579 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:26.702525 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:26.703381 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:26.703921 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:26.704085 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:26.706433 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:26.706439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:26.706456 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:26.706486 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:26.708884 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:26.709325 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:26.709331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:26.709345 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:26.709482 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:26.709730 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:26.709733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:26.709735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:26.710244 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:26.710449 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:26.710653 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 03:01:26.710944 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":153,"threads_started":1}
I20260811 03:01:26.712558 10267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01685436cd444e3fa9bbdc8218f8c84e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33941 } }
I20260811 03:01:26.712714 10257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:26.714053 10272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01685436cd444e3fa9bbdc8218f8c84e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01685436cd444e3fa9bbdc8218f8c84e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33941 } } }
I20260811 03:01:26.714151 10272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:26.714527 10270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01685436cd444e3fa9bbdc8218f8c84e. Latest consensus state: current_term: 1 leader_uuid: "01685436cd444e3fa9bbdc8218f8c84e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01685436cd444e3fa9bbdc8218f8c84e" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33941 } } }
I20260811 03:01:26.714620 10270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:26.714701 10289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:26.717711 10289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:26.722283 10289 catalog_manager.cc:1347] Generated new cluster ID: 90cbac8d7e23429b810a45e0645f5cf5
I20260811 03:01:26.722353 10289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:26.759059 10289 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:26.759984 10289 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:26.763295 10289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e: Generated new TSK 0
I20260811 03:01:26.764007 10289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 03:01:26.802063 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33941
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:26.913043 10313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:26.913317 10313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:26.913376 10313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:26.917351 10313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:26.917399 10313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:26.917474 10313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:01:26.922447 10313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:26.923368 10313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:26.924310 10313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:01:26.933540 10313 server_base.cc:1034] running on GCE node
W20260811 03:01:26.933504 10328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:26.933502 10327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:26.933544 10330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:26.934144 10313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:26.934721 10313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:26.935883 10313 hybrid_clock.cc:648] HybridClock initialized: now 1786417286935866 us; error 31 us; skew 500 ppm
Aug 11 03:01:26 dist-test-slave-gjw7 krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:26.938589 10313 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:01:26.939399 10313 webserver.cc:460] Webserver started at http://127.8.203.65:38547/ using document root <none> and password file <none>
I20260811 03:01:26.939890 10313 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:26.939946 10313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:26.940107 10313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:26.942056 10313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "b3885bebd5494b2180303aaeaac751a1"
format_stamp: "Formatted at 2026-08-11 03:01:26 on dist-test-slave-gjw7"
server_key: "01423f634a09ba5ed023c367c08b949d"
server_key_iv: "cfbc90e87d59c49b841179835cfd0cc5"
server_key_version: "encryptionkey@0"
I20260811 03:01:26.942562 10313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "b3885bebd5494b2180303aaeaac751a1"
format_stamp: "Formatted at 2026-08-11 03:01:26 on dist-test-slave-gjw7"
server_key: "01423f634a09ba5ed023c367c08b949d"
server_key_iv: "cfbc90e87d59c49b841179835cfd0cc5"
server_key_version: "encryptionkey@0"
I20260811 03:01:26.945964 10313 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:01:26.948153 10344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:26.949250 10313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 03:01:26.949386 10313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "b3885bebd5494b2180303aaeaac751a1"
format_stamp: "Formatted at 2026-08-11 03:01:26 on dist-test-slave-gjw7"
server_key: "01423f634a09ba5ed023c367c08b949d"
server_key_iv: "cfbc90e87d59c49b841179835cfd0cc5"
server_key_version: "encryptionkey@0"
I20260811 03:01:26.949482 10313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:26.968873 10313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:26.971709 10313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:26.971876 10313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:26.972445 10313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:26.973228 10313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:26.973274 10313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:26.973362 10313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:26.973394 10313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:26.981700 10313 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:36743
I20260811 03:01:26.981722 10577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:36743 every 8 connection(s)
I20260811 03:01:26.982543 10313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:01:26.988302 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10313
I20260811 03:01:26.988409 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:01:26.988668 9005 external_mini_cluster.cc:1428] Setting key 2b68154960239074fa09e94deaa1beb7
Aug 11 03:01:26 dist-test-slave-gjw7 krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:26.995221 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:26.984186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:36327 (local address 127.8.203.126:33941)
0811 03:01:26.984326 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:26.984330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:26.985140 (+ 810us) server_negotiation.cc:407] Connection header received
0811 03:01:26.986045 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:26.986048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:26.986104 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:26.986178 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:26.987313 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:26.987851 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:26.989034 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:26.989256 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:26.989400 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:26.991862 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:26.991867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:26.991869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:26.991898 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:26.993320 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:26.993779 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:26.993783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:26.993785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:26.993878 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:26.994165 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:26.994168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:26.994170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:26.994388 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:26.994483 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:26.994925 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 03:01:26.995055 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 03:01:26.995886 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:26.984528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33941 (local address 127.8.203.65:36327)
0811 03:01:26.984999 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:26.985041 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:26.985848 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:26.986368 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:26.986375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:26.986704 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:26.987169 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:26.987182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:26.987972 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:26.987976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:26.988918 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:26.988929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:26.989448 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:26.989452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:26.989607 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:26.990110 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:26.990126 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:26.991701 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:26.993433 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:26.993437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:26.993448 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:26.993683 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:26.993961 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:26.993963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:26.993964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:26.994087 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:26.994476 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:26.994481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:26.994693 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 03:01:26.994835 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 03:01:26.995073 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
I20260811 03:01:26.997036 10579 heartbeater.cc:347] Connected to a master server at 127.8.203.126:33941
I20260811 03:01:26.997239 10579 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:26.997692 10579 heartbeater.cc:511] Master 127.8.203.126:33941 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 03:01:26.999107 10184 ts_manager.cc:194] Registered new tserver with Master: b3885bebd5494b2180303aaeaac751a1 (127.8.203.65:36743)
I20260811 03:01:27.000186 10184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:36327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 03:01:27.013954 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33941
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:27.122635 10588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:27.122900 10588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:27.122960 10588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:27.126941 10588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:27.126997 10588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:27.127076 10588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:01:27.132184 10588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:27.133159 10588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:27.134141 10588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:27.143210 10607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:27.143203 10599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:27.143206 10601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:27.143618 10588 server_base.cc:1034] running on GCE node
I20260811 03:01:27.144006 10588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:27.144594 10588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:27.145754 10588 hybrid_clock.cc:648] HybridClock initialized: now 1786417287145732 us; error 37 us; skew 500 ppm
Aug 11 03:01:27 dist-test-slave-gjw7 krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417287, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:27.148259 10588 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:01:27.149116 10588 webserver.cc:460] Webserver started at http://127.8.203.66:41301/ using document root <none> and password file <none>
I20260811 03:01:27.149682 10588 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:27.149749 10588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:27.149967 10588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:27.151952 10588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "82fb886ac3d34d4b860f900a96ff060f"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "1fb64f9e06b142edd3819b52f4310c3b"
server_key_iv: "bb56453b2706d9af926c867bd5ab5f8a"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.152489 10588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "82fb886ac3d34d4b860f900a96ff060f"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "1fb64f9e06b142edd3819b52f4310c3b"
server_key_iv: "bb56453b2706d9af926c867bd5ab5f8a"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.156065 10588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 03:01:27.158447 10629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:27.159586 10588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:27.159694 10588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "82fb886ac3d34d4b860f900a96ff060f"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "1fb64f9e06b142edd3819b52f4310c3b"
server_key_iv: "bb56453b2706d9af926c867bd5ab5f8a"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.159783 10588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:27.172708 10588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:27.175242 10588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:27.175405 10588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:27.175958 10588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:27.176735 10588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:27.176780 10588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:27.176831 10588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:27.176868 10588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:27.185068 10588 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:46237
I20260811 03:01:27.185091 10875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:46237 every 8 connection(s)
I20260811 03:01:27.185918 10588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:01:27.189379 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10588
I20260811 03:01:27.189500 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:01:27.189810 9005 external_mini_cluster.cc:1428] Setting key 359c65b42c9b68c7f9abb178de1b2611
Aug 11 03:01:27 dist-test-slave-gjw7 krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417287, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:27.199301 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.187598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:47765 (local address 127.8.203.126:33941)
0811 03:01:27.187732 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.187737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.188585 (+ 848us) server_negotiation.cc:407] Connection header received
0811 03:01:27.189429 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.189433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.189494 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.189583 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:27.190799 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.191455 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.192923 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.193129 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.193212 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.195518 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:27.195522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:27.195525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:27.195551 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:27.197221 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:27.197722 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:27.197726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:27.197728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:27.197822 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:27.198112 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:27.198115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:27.198117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:27.198350 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:27.198454 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.198993 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.199124 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 03:01:27.200028 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.187990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33941 (local address 127.8.203.66:47765)
0811 03:01:27.188440 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.188480 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:27.189212 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:27.189756 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:27.189763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:27.190122 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:27.190660 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.190675 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.191592 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.191596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.192810 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.192822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.193271 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.193276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.193488 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.193998 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:27.194014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:27.195348 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:27.197356 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:27.197363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:27.197378 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:27.197627 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:27.197911 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:27.197913 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:27.197915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:27.198033 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:27.198449 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:27.198454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:27.198735 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 03:01:27.198916 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 03:01:27.199177 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 03:01:27.201188 10876 heartbeater.cc:347] Connected to a master server at 127.8.203.126:33941
I20260811 03:01:27.201447 10876 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:27.201975 10876 heartbeater.cc:511] Master 127.8.203.126:33941 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
I20260811 03:01:27.203037 10184 ts_manager.cc:194] Registered new tserver with Master: 82fb886ac3d34d4b860f900a96ff060f (127.8.203.66:46237)
I20260811 03:01:27.203579 10184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:47765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 03:01:27.215876 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33941
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:27.325199 10882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:27.325465 10882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:27.325507 10882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:27.329439 10882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:27.329493 10882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:27.329550 10882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:01:27.334466 10882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:33941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:27.335326 10882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:27.336237 10882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:27.345216 10891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:27.345229 10890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:27.345224 10894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:27.345461 10882 server_base.cc:1034] running on GCE node
I20260811 03:01:27.345925 10882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:27.346454 10882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:27.347605 10882 hybrid_clock.cc:648] HybridClock initialized: now 1786417287347586 us; error 37 us; skew 500 ppm
Aug 11 03:01:27 dist-test-slave-gjw7 krb5kdc[10084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417287, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:27.350204 10882 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:01:27.350997 10882 webserver.cc:460] Webserver started at http://127.8.203.67:34345/ using document root <none> and password file <none>
I20260811 03:01:27.351478 10882 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:27.351532 10882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:27.351692 10882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:27.353735 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "3764bd228bd34c1dab748e91935384a7"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "ffdf40e3963ce9ab1ef8567b5ec914b4"
server_key_iv: "042937c2b5d9599c0da71e4a1295c7db"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.354221 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "3764bd228bd34c1dab748e91935384a7"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "ffdf40e3963ce9ab1ef8567b5ec914b4"
server_key_iv: "042937c2b5d9599c0da71e4a1295c7db"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.357695 10882 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:01:27.359869 10903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:27.361011 10882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:27.361102 10882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "3764bd228bd34c1dab748e91935384a7"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "ffdf40e3963ce9ab1ef8567b5ec914b4"
server_key_iv: "042937c2b5d9599c0da71e4a1295c7db"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.361194 10882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:27.379590 10882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:27.382496 10882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:27.382653 10882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:27.383174 10882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:27.383948 10882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:27.383993 10882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:27.384033 10882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:27.384063 10882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:27.392364 10882 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:39257
I20260811 03:01:27.392413 11132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:39257 every 8 connection(s)
I20260811 03:01:27.393229 10882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
Aug 11 03:01:27 dist-test-slave-gjw7 krb5kdc[10084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417287, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:27.401451 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 10882
I20260811 03:01:27.401543 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:01:27.401860 9005 external_mini_cluster.cc:1428] Setting key d5f56ac9bc16c38134d27c5174e33e9e
I20260811 03:01:27.405094 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.394900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:46681 (local address 127.8.203.126:33941)
0811 03:01:27.395033 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.395037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.395799 (+ 762us) server_negotiation.cc:407] Connection header received
0811 03:01:27.396660 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.396663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.396720 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.396791 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:27.397901 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.398417 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.399503 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.399714 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.399801 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.402136 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:27.402139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:27.402141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:27.402164 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:27.403434 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:27.403926 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:27.403929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:27.403931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:27.404010 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:27.404267 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:27.404269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:27.404270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:27.404430 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:27.404502 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.404888 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.404980 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 03:01:27.405830 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.395233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:33941 (local address 127.8.203.67:46681)
0811 03:01:27.395658 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.395698 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:27.396461 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:27.396966 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:27.396973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:27.397306 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:27.397752 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.397764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.398535 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.398537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.399385 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.399395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.399847 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.399852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.400005 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.400465 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:27.400481 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:27.401971 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:27.403546 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:27.403552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:27.403565 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:27.403835 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:27.404079 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:27.404081 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:27.404082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:27.404200 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:27.404501 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:27.404505 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:27.404711 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 03:01:27.404856 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 03:01:27.405081 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":120,"threads_started":1}
I20260811 03:01:27.406769 11133 heartbeater.cc:347] Connected to a master server at 127.8.203.126:33941
I20260811 03:01:27.406977 11133 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:27.407397 11133 heartbeater.cc:511] Master 127.8.203.126:33941 requested a full tablet report, sending...
I20260811 03:01:27.408275 10184 ts_manager.cc:194] Registered new tserver with Master: 3764bd228bd34c1dab748e91935384a7 (127.8.203.67:39257)
I20260811 03:01:27.408812 10184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:46681
I20260811 03:01:27.415169 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:01:27.422932 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.416955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.8.203.126:33941)
0811 03:01:27.417076 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.417080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.417215 (+ 135us) server_negotiation.cc:407] Connection header received
0811 03:01:27.417328 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.417330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.417367 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.417436 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:27.418031 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.418496 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.419234 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.419441 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.419512 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.420566 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:27.420569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:27.420571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:27.420591 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:27.421767 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:27.422086 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:27.422088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:27.422089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:27.422150 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:27.422321 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:27.422323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:27.422324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:27.422497 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:27.422592 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.422739 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.422820 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 03:01:27.425377 10184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:01:27.428206 10184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:01:27.441504 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.435730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:36743 (local address 127.0.0.1:48806)
0811 03:01:27.436328 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.436388 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:27.436506 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:27.437179 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:27.437185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:27.437210 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:27.437355 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.437363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.438577 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.438580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.439882 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.439893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.440897 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.440901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.441005 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.441039 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 03:01:27.441155 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 03:01:27.441338 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":20,"thread_start_us":98,"threads_started":1}
I20260811 03:01:27.441592 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.435715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.8.203.67:39257)
0811 03:01:27.436048 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.436059 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.436639 (+ 580us) server_negotiation.cc:407] Connection header received
0811 03:01:27.436717 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.436722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.437141 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.437279 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:27.437563 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.438710 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.440054 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.440808 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.440912 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.441047 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.441367 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.441457 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":132,"threads_started":1}
I20260811 03:01:27.441756 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.436785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55420 (local address 127.8.203.66:46237)
0811 03:01:27.437355 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.437366 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.437397 (+ 31us) server_negotiation.cc:407] Connection header received
0811 03:01:27.437465 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.437468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.437622 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.437767 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:27.437976 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.438979 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.440211 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.440981 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.441110 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.441232 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.441531 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.441629 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":106,"threads_started":1}
I20260811 03:01:27.441649 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.436247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:46237 (local address 127.0.0.1:55420)
0811 03:01:27.437002 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.437016 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:27.437095 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:27.437752 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:27.437754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:27.437773 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:27.437865 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.437871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.439091 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.439094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.440092 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.440103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.441124 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.441128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.441221 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.441250 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 03:01:27.441365 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 03:01:27.441439 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":4,"thread_start_us":86,"threads_started":1}
I20260811 03:01:27.441735 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.435581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:39257 (local address 127.0.0.1:60702)
0811 03:01:27.436465 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.436477 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:27.436555 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:27.437368 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:27.437371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:27.437388 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:27.437463 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.437469 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.438846 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.438850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.439937 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.439945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.441109 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.441113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.441196 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.441241 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 03:01:27.441329 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 03:01:27.441394 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":3,"thread_start_us":66,"threads_started":1}
I20260811 03:01:27.442291 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.436247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.8.203.65:36743)
0811 03:01:27.436698 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.436709 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.436730 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:01:27.436805 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.436810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.437006 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.437510 (+ 504us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:27.437648 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.438459 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.439992 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.440598 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.440886 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.441034 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.442088 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.442179 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":152,"threads_started":1}
I20260811 03:01:27.444630 10441 tablet_service.cc:1467] Processing CreateTablet for tablet a93a29f03c4f444ca9be6a0de1c601d5 (DEFAULT_TABLE table=test-table [id=fcf78ad6ab8845fea6da18524cdd183c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:27.444679 11003 tablet_service.cc:1467] Processing CreateTablet for tablet a93a29f03c4f444ca9be6a0de1c601d5 (DEFAULT_TABLE table=test-table [id=fcf78ad6ab8845fea6da18524cdd183c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:27.444679 10740 tablet_service.cc:1467] Processing CreateTablet for tablet a93a29f03c4f444ca9be6a0de1c601d5 (DEFAULT_TABLE table=test-table [id=fcf78ad6ab8845fea6da18524cdd183c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:27.445648 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93a29f03c4f444ca9be6a0de1c601d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:27.445696 10740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93a29f03c4f444ca9be6a0de1c601d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:27.445703 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a93a29f03c4f444ca9be6a0de1c601d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:27.451712 11158 tablet_bootstrap.cc:492] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1: Bootstrap starting.
I20260811 03:01:27.453090 11160 tablet_bootstrap.cc:492] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Bootstrap starting.
I20260811 03:01:27.453090 11166 tablet_bootstrap.cc:492] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Bootstrap starting.
I20260811 03:01:27.454108 11158 tablet_bootstrap.cc:654] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:27.455088 11158 log.cc:824] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:27.455176 11160 tablet_bootstrap.cc:654] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:27.455574 11166 tablet_bootstrap.cc:654] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:27.456411 11166 log.cc:824] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:27.456411 11160 log.cc:824] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:27.457376 11158 tablet_bootstrap.cc:492] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1: No bootstrap required, opened a new log
I20260811 03:01:27.457557 11158 ts_tablet_manager.cc:1397] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 03:01:27.458388 11160 tablet_bootstrap.cc:492] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: No bootstrap required, opened a new log
I20260811 03:01:27.458388 11166 tablet_bootstrap.cc:492] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: No bootstrap required, opened a new log
I20260811 03:01:27.458570 11160 ts_tablet_manager.cc:1397] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:01:27.458570 11166 ts_tablet_manager.cc:1397] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:01:27.460335 11158 raft_consensus.cc:348] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.460541 11158 raft_consensus.cc:374] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:27.460611 11158 raft_consensus.cc:729] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3885bebd5494b2180303aaeaac751a1, State: Initialized, Role: FOLLOWER
I20260811 03:01:27.461023 11158 consensus_queue.cc:260] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.461624 10579 heartbeater.cc:503] Master 127.8.203.126:33941 was elected leader, sending a full tablet report...
I20260811 03:01:27.462234 11160 raft_consensus.cc:348] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.462234 11166 raft_consensus.cc:348] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.462494 11166 raft_consensus.cc:374] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:27.462592 11160 raft_consensus.cc:374] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:27.462634 11166 raft_consensus.cc:729] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fb886ac3d34d4b860f900a96ff060f, State: Initialized, Role: FOLLOWER
I20260811 03:01:27.462713 11160 raft_consensus.cc:729] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3764bd228bd34c1dab748e91935384a7, State: Initialized, Role: FOLLOWER
I20260811 03:01:27.462867 11158 ts_tablet_manager.cc:1428] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:01:27.463155 11160 consensus_queue.cc:260] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.463383 11166 consensus_queue.cc:260] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.464304 11166 ts_tablet_manager.cc:1428] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 03:01:27.464308 11133 heartbeater.cc:503] Master 127.8.203.126:33941 was elected leader, sending a full tablet report...
I20260811 03:01:27.464359 11160 ts_tablet_manager.cc:1428] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 03:01:27.465330 10876 heartbeater.cc:503] Master 127.8.203.126:33941 was elected leader, sending a full tablet report...
W20260811 03:01:27.483628 10582 tablet.cc:2369] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:01:27.556229 11182 raft_consensus.cc:482] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:27.556418 11182 raft_consensus.cc:504] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.557585 11182 leader_election.cc:288] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3764bd228bd34c1dab748e91935384a7 (127.8.203.67:39257), b3885bebd5494b2180303aaeaac751a1 (127.8.203.65:36743)
I20260811 03:01:27.562183 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.557907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:34625 (local address 127.8.203.67:39257)
0811 03:01:27.558046 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.558051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.558072 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:01:27.558128 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.558131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.558181 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.558244 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:27.558543 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.559295 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.560848 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.561478 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.561588 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.561641 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.561990 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.562056 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:01:27.562182 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.558048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:45363 (local address 127.8.203.65:36743)
0811 03:01:27.558180 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.558184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.558368 (+ 184us) server_negotiation.cc:407] Connection header received
0811 03:01:27.558462 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.558464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.558511 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.558586 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:27.558805 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.559479 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.560837 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.561470 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.561588 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.561641 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.561990 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.562056 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:01:27.562273 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.557809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:39257 (local address 127.8.203.66:34625)
0811 03:01:27.557926 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.557941 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:27.558030 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:27.558277 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:27.558280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:27.558302 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:27.558435 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.558442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.559405 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.559407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.560711 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.560724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.561584 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.561588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.561834 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.561891 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 03:01:27.561970 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 03:01:27.562098 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":161}
I20260811 03:01:27.562281 11184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.557816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:36743 (local address 127.8.203.66:45363)
0811 03:01:27.558279 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.558291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:27.558379 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:27.558600 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:27.558602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:27.558618 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:27.558717 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.558721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.559564 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.559567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.560733 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:27.560740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.561584 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.561588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:27.561830 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.561896 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 03:01:27.561970 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 03:01:27.562099 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":209,"thread_start_us":142,"threads_started":1}
I20260811 03:01:27.562966 11048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93a29f03c4f444ca9be6a0de1c601d5" candidate_uuid: "82fb886ac3d34d4b860f900a96ff060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3764bd228bd34c1dab748e91935384a7" is_pre_election: true
I20260811 03:01:27.563110 10488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93a29f03c4f444ca9be6a0de1c601d5" candidate_uuid: "82fb886ac3d34d4b860f900a96ff060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3885bebd5494b2180303aaeaac751a1" is_pre_election: true
I20260811 03:01:27.563287 11048 raft_consensus.cc:2454] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82fb886ac3d34d4b860f900a96ff060f in term 0.
I20260811 03:01:27.563426 10488 raft_consensus.cc:2454] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82fb886ac3d34d4b860f900a96ff060f in term 0.
I20260811 03:01:27.563863 10634 leader_election.cc:302] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3764bd228bd34c1dab748e91935384a7, 82fb886ac3d34d4b860f900a96ff060f; no voters:
I20260811 03:01:27.564152 11182 raft_consensus.cc:2781] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:01:27.564242 11182 raft_consensus.cc:482] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:01:27.564293 11182 raft_consensus.cc:3037] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:27.565375 11182 raft_consensus.cc:504] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.565683 11182 leader_election.cc:288] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [CANDIDATE]: Term 1 election: Requested vote from peers 3764bd228bd34c1dab748e91935384a7 (127.8.203.67:39257), b3885bebd5494b2180303aaeaac751a1 (127.8.203.65:36743)
I20260811 03:01:27.566008 10488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93a29f03c4f444ca9be6a0de1c601d5" candidate_uuid: "82fb886ac3d34d4b860f900a96ff060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3885bebd5494b2180303aaeaac751a1"
I20260811 03:01:27.566027 11048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a93a29f03c4f444ca9be6a0de1c601d5" candidate_uuid: "82fb886ac3d34d4b860f900a96ff060f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3764bd228bd34c1dab748e91935384a7"
I20260811 03:01:27.566126 11048 raft_consensus.cc:3037] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:27.566126 10488 raft_consensus.cc:3037] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:27.567327 11048 raft_consensus.cc:2454] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82fb886ac3d34d4b860f900a96ff060f in term 1.
I20260811 03:01:27.567327 10488 raft_consensus.cc:2454] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82fb886ac3d34d4b860f900a96ff060f in term 1.
I20260811 03:01:27.567620 10637 leader_election.cc:302] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82fb886ac3d34d4b860f900a96ff060f, b3885bebd5494b2180303aaeaac751a1; no voters:
I20260811 03:01:27.567773 11182 raft_consensus.cc:2781] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:27.568003 11182 raft_consensus.cc:686] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 1 LEADER]: Becoming Leader. State: Replica: 82fb886ac3d34d4b860f900a96ff060f, State: Running, Role: LEADER
I20260811 03:01:27.568328 11182 consensus_queue.cc:237] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } }
I20260811 03:01:27.571323 10182 catalog_manager.cc:5495] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f reported cstate change: term changed from 0 to 1, leader changed from <none> to 82fb886ac3d34d4b860f900a96ff060f (127.8.203.66). New cstate: current_term: 1 leader_uuid: "82fb886ac3d34d4b860f900a96ff060f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82fb886ac3d34d4b860f900a96ff060f" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46237 } health_report { overall_health: HEALTHY } } }
I20260811 03:01:27.588320 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.585450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.8.203.66:46237)
0811 03:01:27.585569 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.585572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.585584 (+ 12us) server_negotiation.cc:407] Connection header received
0811 03:01:27.585694 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.585696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.585745 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.585823 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:01:27.585999 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.586457 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.587197 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.587385 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.587449 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.587646 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:01:27.588018 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:01:27.588101 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.588131 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.588175 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 03:01:27.593608 10488 raft_consensus.cc:1261] T a93a29f03c4f444ca9be6a0de1c601d5 P b3885bebd5494b2180303aaeaac751a1 [term 1 FOLLOWER]: Refusing update from remote peer 82fb886ac3d34d4b860f900a96ff060f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:27.593608 11048 raft_consensus.cc:1261] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 1 FOLLOWER]: Refusing update from remote peer 82fb886ac3d34d4b860f900a96ff060f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:27.594282 11182 consensus_queue.cc:1035] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3885bebd5494b2180303aaeaac751a1" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:27.594477 11185 consensus_queue.cc:1035] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3764bd228bd34c1dab748e91935384a7" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 39257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:27.600914 11188 mvcc.cc:205] Tried to move back new op lower bound from 7317165209976434688 to 7317165209882382336. Current Snapshot: MvccSnapshot[applied={T|T < 7317165209976434688}]
I20260811 03:01:27.601929 11189 mvcc.cc:205] Tried to move back new op lower bound from 7317165209976434688 to 7317165209882382336. Current Snapshot: MvccSnapshot[applied={T|T < 7317165209976434688}]
I20260811 03:01:27.611012 10179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:27.611147 10179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:27.613268 10179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01685436cd444e3fa9bbdc8218f8c84e: Sending DeleteTablet for 3 replicas of tablet a93a29f03c4f444ca9be6a0de1c601d5
I20260811 03:01:27.614084 11003 tablet_service.cc:1514] Processing DeleteTablet for tablet a93a29f03c4f444ca9be6a0de1c601d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:27 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:60702
I20260811 03:01:27.614172 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet a93a29f03c4f444ca9be6a0de1c601d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:27 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:48806
I20260811 03:01:27.614226 10740 tablet_service.cc:1514] Processing DeleteTablet for tablet a93a29f03c4f444ca9be6a0de1c601d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:27 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:55420
I20260811 03:01:27.614687 11204 tablet_replica.cc:330] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: stopping tablet replica
I20260811 03:01:27.614756 11205 tablet_replica.cc:330] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: stopping tablet replica
I20260811 03:01:27.615087 11205 raft_consensus.cc:2229] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:01:27.615095 11204 raft_consensus.cc:2229] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:27.615123 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10313
I20260811 03:01:27.615736 11204 raft_consensus.cc:2258] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:27.619062 11205 raft_consensus.cc:2258] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:27.626736 11204 ts_tablet_manager.cc:1905] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:27.626736 11205 ts_tablet_manager.cc:1905] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:27.629489 11204 ts_tablet_manager.cc:1918] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:27.629489 11205 ts_tablet_manager.cc:1918] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:27.629588 11205 log.cc:1196] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/a93a29f03c4f444ca9be6a0de1c601d5
I20260811 03:01:27.629667 11204 log.cc:1196] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/a93a29f03c4f444ca9be6a0de1c601d5
I20260811 03:01:27.629997 11205 ts_tablet_manager.cc:1939] T a93a29f03c4f444ca9be6a0de1c601d5 P 82fb886ac3d34d4b860f900a96ff060f: Deleting consensus metadata
I20260811 03:01:27.630043 11204 ts_tablet_manager.cc:1939] T a93a29f03c4f444ca9be6a0de1c601d5 P 3764bd228bd34c1dab748e91935384a7: Deleting consensus metadata
I20260811 03:01:27.631253 10146 catalog_manager.cc:4841] TS 3764bd228bd34c1dab748e91935384a7 (127.8.203.67:39257): tablet a93a29f03c4f444ca9be6a0de1c601d5 (table test-table [id=fcf78ad6ab8845fea6da18524cdd183c]) successfully deleted
I20260811 03:01:27.631487 10146 catalog_manager.cc:4841] TS 82fb886ac3d34d4b860f900a96ff060f (127.8.203.66:46237): tablet a93a29f03c4f444ca9be6a0de1c601d5 (table test-table [id=fcf78ad6ab8845fea6da18524cdd183c]) successfully deleted
W20260811 03:01:27.632579 10149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:36743 (error 108)
I20260811 03:01:27.633450 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10588
I20260811 03:01:27.633580 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.633278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:36743 (local address 127.0.0.1:48818)
0811 03:01:27.633428 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:27.633499 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 03:01:27.645794 10149 catalog_manager.cc:4584] TS b3885bebd5494b2180303aaeaac751a1 (127.8.203.65:36743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a93a29f03c4f444ca9be6a0de1c601d5: Network error: Client connection negotiation failed: client connection to 127.8.203.65:36743: connect: Connection refused (error 111)
I20260811 03:01:27.651039 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10882
I20260811 03:01:27.662830 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 10115
2026-08-11T03:01:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2469 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T03:01:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:27Z Disabled control of system clock
I20260811 03:01:27.697523 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:34627
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:42817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:34627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:01:27.809574 11213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:27.809818 11213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:27.809864 11213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:27.813730 11213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:27.813779 11213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:27.813799 11213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:27.813834 11213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:27.813865 11213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:27.818532 11213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:34627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:34627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:27.819584 11213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:27.820887 11213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:27.828790 11232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:27.828928 11213 server_base.cc:1034] running on GCE node
W20260811 03:01:27.828799 11234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:27.828814 11227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:27.829624 11213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:27.830614 11213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:27.831770 11213 hybrid_clock.cc:648] HybridClock initialized: now 1786417287831752 us; error 28 us; skew 500 ppm
I20260811 03:01:27.833568 11213 webserver.cc:460] Webserver started at http://127.8.203.126:39497/ using document root <none> and password file <none>
I20260811 03:01:27.834096 11213 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:27.834160 11213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:27.834363 11213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:27.836269 11213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "b19ebf72d60d4575a8e921d6bbafa890"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "168704c4232788b1cd0ebeba3d8a1b5a"
server_key_iv: "485feb05047ea39cb3f0571a868ed5c6"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.836789 11213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "b19ebf72d60d4575a8e921d6bbafa890"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "168704c4232788b1cd0ebeba3d8a1b5a"
server_key_iv: "485feb05047ea39cb3f0571a868ed5c6"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.840186 11213 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:01:27.842442 11246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:27.843532 11213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:27.843640 11213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "b19ebf72d60d4575a8e921d6bbafa890"
format_stamp: "Formatted at 2026-08-11 03:01:27 on dist-test-slave-gjw7"
server_key: "168704c4232788b1cd0ebeba3d8a1b5a"
server_key_iv: "485feb05047ea39cb3f0571a868ed5c6"
server_key_version: "encryptionkey@0"
I20260811 03:01:27.843740 11213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:27.860282 11213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:27.860849 11213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:27.861011 11213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:27.867324 11213 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:34627
I20260811 03:01:27.867355 11338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:34627 every 8 connection(s)
I20260811 03:01:27.868166 11213 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:27.871152 11340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:27.872732 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11213
I20260811 03:01:27.872829 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:27.873080 9005 external_mini_cluster.cc:1428] Setting key 3cad2eee090da29be724949017a03170
I20260811 03:01:27.877262 11340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890: Bootstrap starting.
I20260811 03:01:27.880028 11340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:27.880927 11340 log.cc:824] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:27.883744 11340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890: No bootstrap required, opened a new log
I20260811 03:01:27.884629 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:27.874457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.8.203.126:34627)
0811 03:01:27.875107 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:27.875127 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:27.875171 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:01:27.876200 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:27.876226 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:27.876724 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:27.877064 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:27.877529 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.878950 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.880253 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:27.880947 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:27.881197 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:27.882227 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:27.882234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:27.882251 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:27.882732 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:27.882798 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:27.882824 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:27.882913 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:27.883406 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 03:01:27.883665 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":101,"threads_started":1}
I20260811 03:01:27.887445 11340 raft_consensus.cc:348] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19ebf72d60d4575a8e921d6bbafa890" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 34627 } }
I20260811 03:01:27.887665 11340 raft_consensus.cc:374] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:27.887717 11340 raft_consensus.cc:729] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b19ebf72d60d4575a8e921d6bbafa890, State: Initialized, Role: FOLLOWER
I20260811 03:01:27.888150 11340 consensus_queue.cc:260] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19ebf72d60d4575a8e921d6bbafa890" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 34627 } }
I20260811 03:01:27.888283 11340 raft_consensus.cc:388] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:27.888345 11340 raft_consensus.cc:482] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:27.888440 11340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:27.889266 11340 raft_consensus.cc:504] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19ebf72d60d4575a8e921d6bbafa890" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 34627 } }
I20260811 03:01:27.889618 11340 leader_election.cc:302] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b19ebf72d60d4575a8e921d6bbafa890; no voters:
I20260811 03:01:27.889909 11340 leader_election.cc:288] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:27.890224 11348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:27.890465 11348 raft_consensus.cc:686] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [term 1 LEADER]: Becoming Leader. State: Replica: b19ebf72d60d4575a8e921d6bbafa890, State: Running, Role: LEADER
I20260811 03:01:27.890806 11348 consensus_queue.cc:237] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19ebf72d60d4575a8e921d6bbafa890" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 34627 } }
I20260811 03:01:27.891129 11340 sys_catalog.cc:564] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:27.892076 11348 sys_catalog.cc:455] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b19ebf72d60d4575a8e921d6bbafa890. Latest consensus state: current_term: 1 leader_uuid: "b19ebf72d60d4575a8e921d6bbafa890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19ebf72d60d4575a8e921d6bbafa890" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 34627 } } }
I20260811 03:01:27.892271 11348 sys_catalog.cc:458] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:27.893169 11347 sys_catalog.cc:455] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b19ebf72d60d4575a8e921d6bbafa890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19ebf72d60d4575a8e921d6bbafa890" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 34627 } } }
I20260811 03:01:27.893273 11347 sys_catalog.cc:458] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:27.893929 11352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:27.897078 11352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:27.902598 11352 catalog_manager.cc:1347] Generated new cluster ID: 20444e798de24a709791496034b9e1f9
I20260811 03:01:27.902657 11352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:27.911437 11352 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:27.912510 11352 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:27.919430 11352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890: Generated new TSK 0
I20260811 03:01:27.920118 11352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:27.924461 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:34627
--builtin_ntp_servers=127.8.203.84:42817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:01:28.035768 11384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:28.036016 11384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:28.036052 11384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:28.039932 11384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:28.039980 11384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:28.040030 11384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:01:28.044580 11384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:34627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:28.045557 11384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:28.046844 11384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:28.056036 11396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:28.056011 11398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:28.056077 11384 server_base.cc:1034] running on GCE node
W20260811 03:01:28.056020 11405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:28.056627 11384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:28.057220 11384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:28.058383 11384 hybrid_clock.cc:648] HybridClock initialized: now 1786417288058366 us; error 29 us; skew 500 ppm
I20260811 03:01:28.060154 11384 webserver.cc:460] Webserver started at http://127.8.203.65:43865/ using document root <none> and password file <none>
I20260811 03:01:28.060703 11384 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:28.060770 11384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:28.060981 11384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:28.062981 11384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "e7b1dd2c9e464904b6c18783ccb76818"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "5444ec62dfe9238e25059dc60325f38a"
server_key_iv: "6f8af92dc5fc4ebf2922a62746dc1c19"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.063513 11384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "e7b1dd2c9e464904b6c18783ccb76818"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "5444ec62dfe9238e25059dc60325f38a"
server_key_iv: "6f8af92dc5fc4ebf2922a62746dc1c19"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.067005 11384 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:01:28.069242 11420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.070396 11384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:28.070501 11384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "e7b1dd2c9e464904b6c18783ccb76818"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "5444ec62dfe9238e25059dc60325f38a"
server_key_iv: "6f8af92dc5fc4ebf2922a62746dc1c19"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.070595 11384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:28.083803 11384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:28.084381 11384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:28.084532 11384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:28.085054 11384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:28.085939 11384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:28.085985 11384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.086035 11384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:28.086064 11384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.094280 11384 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:34473
I20260811 03:01:28.094331 11631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:34473 every 8 connection(s)
I20260811 03:01:28.095144 11384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:01:28.101109 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11384
I20260811 03:01:28.101203 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:01:28.101491 9005 external_mini_cluster.cc:1428] Setting key 7e6ec648f5c309a40f2fb7ec290fd9a0
I20260811 03:01:28.103384 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:34627
--builtin_ntp_servers=127.8.203.84:42817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:01:28.104063 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.096864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:55773 (local address 127.8.203.126:34627)
0811 03:01:28.097020 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.097025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.098066 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 03:01:28.099393 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.099395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.099441 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.099507 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:28.100290 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.100808 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.102108 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.102349 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.102429 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.103625 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:28.103629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:28.103631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:28.103644 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:28.103666 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:28.103674 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:28.103738 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.103866 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.103926 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 03:01:28.104835 11637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.097267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:34627 (local address 127.8.203.65:55773)
0811 03:01:28.097932 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.097972 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.099193 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.099636 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.099642 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.099721 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:28.100177 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.100189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.100934 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.100937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.102003 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.102016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.102501 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.102505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.102694 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.103310 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:28.103312 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:28.103350 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:28.103410 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:28.103753 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:28.103759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:28.103764 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.103851 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.104060 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":109,"threads_started":1}
I20260811 03:01:28.106158 11632 heartbeater.cc:347] Connected to a master server at 127.8.203.126:34627
I20260811 03:01:28.106379 11632 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:28.106843 11632 heartbeater.cc:511] Master 127.8.203.126:34627 requested a full tablet report, sending...
I20260811 03:01:28.108248 11275 ts_manager.cc:194] Registered new tserver with Master: e7b1dd2c9e464904b6c18783ccb76818 (127.8.203.65:34473)
I20260811 03:01:28.109370 11275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.65:55773
W20260811 03:01:28.213572 11639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:28.213798 11639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:28.213845 11639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:28.217761 11639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:28.217805 11639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:28.217875 11639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:01:28.222800 11639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:34627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:28.223791 11639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:28.225133 11639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:28.234264 11649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:28.234323 11650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:28.234381 11652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:28.234464 11639 server_base.cc:1034] running on GCE node
I20260811 03:01:28.234907 11639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:28.235502 11639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:28.236650 11639 hybrid_clock.cc:648] HybridClock initialized: now 1786417288236630 us; error 30 us; skew 500 ppm
I20260811 03:01:28.238365 11639 webserver.cc:460] Webserver started at http://127.8.203.66:44107/ using document root <none> and password file <none>
I20260811 03:01:28.238895 11639 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:28.238960 11639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:28.239153 11639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:28.241140 11639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "48219a9b4eac4275adf97f0713d0af50"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "55d7bc59029965859a18bc096c087488"
server_key_iv: "45cd22a86b484df20bab9352facf7d47"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.241735 11639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "48219a9b4eac4275adf97f0713d0af50"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "55d7bc59029965859a18bc096c087488"
server_key_iv: "45cd22a86b484df20bab9352facf7d47"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.245088 11639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:01:28.247406 11665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.248636 11639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:28.248750 11639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "48219a9b4eac4275adf97f0713d0af50"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "55d7bc59029965859a18bc096c087488"
server_key_iv: "45cd22a86b484df20bab9352facf7d47"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.248840 11639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:28.282322 11639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:28.282954 11639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:28.283113 11639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:28.283679 11639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:28.284595 11639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:28.284641 11639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.284703 11639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:28.284729 11639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.293016 11639 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:41883
I20260811 03:01:28.293058 11869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:41883 every 8 connection(s)
I20260811 03:01:28.293962 11639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:01:28.300424 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11639
I20260811 03:01:28.300519 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:01:28.300797 9005 external_mini_cluster.cc:1428] Setting key 7ffd967328b34fafb032962346225ea2
I20260811 03:01:28.302743 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:34627
--builtin_ntp_servers=127.8.203.84:42817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:01:28.302973 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.295680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:46045 (local address 127.8.203.126:34627)
0811 03:01:28.295814 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.295818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.296856 (+ 1038us) server_negotiation.cc:407] Connection header received
0811 03:01:28.298262 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.298266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.298323 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.298399 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:28.299272 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.299804 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.300977 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.301241 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.301380 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.302495 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:28.302498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:28.302500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:28.302512 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:28.302530 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:28.302537 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:28.302592 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.302773 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.302823 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:01:28.303956 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.296082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:34627 (local address 127.8.203.66:46045)
0811 03:01:28.296715 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.296758 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.298049 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.298584 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.298591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.298673 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:28.299129 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.299143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.299927 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.299930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.300860 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.300873 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.301420 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.301424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.301604 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.302208 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:28.302210 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:28.302263 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:28.302340 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:28.302614 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:28.302623 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:28.302630 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.302773 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.303047 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
I20260811 03:01:28.305097 11871 heartbeater.cc:347] Connected to a master server at 127.8.203.126:34627
I20260811 03:01:28.305321 11871 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:28.305794 11871 heartbeater.cc:511] Master 127.8.203.126:34627 requested a full tablet report, sending...
I20260811 03:01:28.306772 11275 ts_manager.cc:194] Registered new tserver with Master: 48219a9b4eac4275adf97f0713d0af50 (127.8.203.66:41883)
I20260811 03:01:28.307483 11275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.66:46045
W20260811 03:01:28.412439 11876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:28.412654 11876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:28.412688 11876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:28.416532 11876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:28.416580 11876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:28.416628 11876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:01:28.421228 11876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:34627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:28.422156 11876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:28.423424 11876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:28.432636 11894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:28.432621 11888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:28.432639 11886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:28.432722 11876 server_base.cc:1034] running on GCE node
I20260811 03:01:28.433189 11876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:28.433790 11876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:28.434937 11876 hybrid_clock.cc:648] HybridClock initialized: now 1786417288434917 us; error 28 us; skew 500 ppm
I20260811 03:01:28.436602 11876 webserver.cc:460] Webserver started at http://127.8.203.67:36797/ using document root <none> and password file <none>
I20260811 03:01:28.437116 11876 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:28.437175 11876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:28.437433 11876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:28.439435 11876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "750c3f3e035942d68e283bdec3092ef8"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "fab2837e9f19938a35d9f23ce71a01c7"
server_key_iv: "1ff381cd6d1eb5708f0abd953d790cd0"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.439960 11876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "750c3f3e035942d68e283bdec3092ef8"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "fab2837e9f19938a35d9f23ce71a01c7"
server_key_iv: "1ff381cd6d1eb5708f0abd953d790cd0"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.443468 11876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 03:01:28.445732 11906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.446890 11876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:01:28.446992 11876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "750c3f3e035942d68e283bdec3092ef8"
format_stamp: "Formatted at 2026-08-11 03:01:28 on dist-test-slave-gjw7"
server_key: "fab2837e9f19938a35d9f23ce71a01c7"
server_key_iv: "1ff381cd6d1eb5708f0abd953d790cd0"
server_key_version: "encryptionkey@0"
I20260811 03:01:28.447085 11876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:28.482486 11876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:28.483179 11876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:28.483357 11876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:28.483963 11876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:28.484933 11876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:28.484982 11876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.485036 11876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:28.485062 11876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:28.493455 11876 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:34111
I20260811 03:01:28.493501 12151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:34111 every 8 connection(s)
I20260811 03:01:28.494364 11876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:01:28.499979 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 11876
I20260811 03:01:28.500073 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:01:28.500325 9005 external_mini_cluster.cc:1428] Setting key d098a954b533b9a01ff3d816cd302bed
I20260811 03:01:28.503377 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.496096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:41657 (local address 127.8.203.126:34627)
0811 03:01:28.496223 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.496227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.497251 (+ 1024us) server_negotiation.cc:407] Connection header received
0811 03:01:28.498651 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.498654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.498710 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.498793 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:28.499663 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.500256 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.501695 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.501918 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.502001 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.503014 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:28.503017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:28.503019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:28.503031 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:28.503051 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:28.503058 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:28.503115 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.503216 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.503264 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 03:01:28.504168 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.496480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:34627 (local address 127.8.203.67:41657)
0811 03:01:28.497110 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.497152 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.498444 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.498971 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.498978 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.499060 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:28.499529 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.499541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.500371 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.500374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.501588 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.501599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.502056 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.502061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.502265 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.502786 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:28.502788 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:28.502826 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:28.502885 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:28.503121 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:28.503125 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:28.503129 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.503210 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.503430 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
I20260811 03:01:28.505134 12152 heartbeater.cc:347] Connected to a master server at 127.8.203.126:34627
I20260811 03:01:28.505370 12152 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:28.505844 12152 heartbeater.cc:511] Master 127.8.203.126:34627 requested a full tablet report, sending...
I20260811 03:01:28.506774 11275 ts_manager.cc:194] Registered new tserver with Master: 750c3f3e035942d68e283bdec3092ef8 (127.8.203.67:34111)
I20260811 03:01:28.507300 11275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.67:41657
I20260811 03:01:28.513778 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:01:28.518754 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.515561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.8.203.126:34627)
0811 03:01:28.515681 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.515686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.515846 (+ 160us) server_negotiation.cc:407] Connection header received
0811 03:01:28.515953 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.515956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.515995 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.516054 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:28.516259 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.516716 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.517507 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.517706 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.517785 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.518442 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:28.518445 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:28.518447 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:28.518455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:28.518469 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:28.518475 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:28.518519 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.518577 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.518619 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:01:28.521231 11275 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:01:28.524117 11275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:01:28.537750 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.532194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:34111 (local address 127.0.0.1:46910)
0811 03:01:28.533119 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.533131 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.533200 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.533664 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.533670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.533703 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:28.533972 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.533982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.534934 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.534936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.536138 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.536149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.537190 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.537194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.537324 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.537364 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.537471 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.537593 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":96,"spinlock_wait_cycles":2304,"thread_start_us":96,"threads_started":1}
I20260811 03:01:28.537770 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.532590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.8.203.67:34111)
0811 03:01:28.533007 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.533016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.533307 (+ 291us) server_negotiation.cc:407] Connection header received
0811 03:01:28.533378 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.533382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.533522 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.533655 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:28.534095 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.534850 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.536262 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.537066 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.537172 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.537399 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.537518 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.537611 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":97,"threads_started":1}
I20260811 03:01:28.538535 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.531963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34473 (local address 127.0.0.1:51224)
0811 03:01:28.533056 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.533069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.533171 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.533801 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.533804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.533819 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:28.533980 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.533987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.536036 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.536039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.537156 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.537167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.538166 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.538169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.538256 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.538280 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.538343 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.538409 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"mutex_wait_us":90,"thread_start_us":108,"threads_started":1}
I20260811 03:01:28.538729 12168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.531818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:41883 (local address 127.0.0.1:39186)
0811 03:01:28.532557 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.532604 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.532731 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.533677 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.533680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.533707 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:28.533972 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.533982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.536546 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.536550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.537431 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.537441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.538405 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.538408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.538488 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.538508 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.538552 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.538610 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":95,"spinlock_wait_cycles":1280,"thread_start_us":58,"threads_started":1}
I20260811 03:01:28.539345 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.532471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.8.203.65:34473)
0811 03:01:28.532917 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.532930 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.533159 (+ 229us) server_negotiation.cc:407] Connection header received
0811 03:01:28.533282 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.533305 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.533520 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.533666 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:28.534968 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.535937 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.537278 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.538048 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.538155 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.538332 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.539095 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.539190 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":119,"threads_started":1}
I20260811 03:01:28.539366 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.531962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39186 (local address 127.8.203.66:41883)
0811 03:01:28.533213 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.533224 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.533242 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:01:28.533323 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.533327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.533520 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.533889 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:28.535440 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.536453 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.537556 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.538276 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.538896 (+ 620us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.539073 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.539123 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.539207 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1151,"thread_start_us":122,"threads_started":1}
I20260811 03:01:28.541090 12013 tablet_service.cc:1467] Processing CreateTablet for tablet 89f4d5f38cbe4c28bd106296d1276271 (DEFAULT_TABLE table=test-table [id=4221fba1867e415f84267c33cb411b5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:28.541867 11501 tablet_service.cc:1467] Processing CreateTablet for tablet 89f4d5f38cbe4c28bd106296d1276271 (DEFAULT_TABLE table=test-table [id=4221fba1867e415f84267c33cb411b5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:28.542275 12013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f4d5f38cbe4c28bd106296d1276271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:28.542866 11501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f4d5f38cbe4c28bd106296d1276271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:28.542695 11748 tablet_service.cc:1467] Processing CreateTablet for tablet 89f4d5f38cbe4c28bd106296d1276271 (DEFAULT_TABLE table=test-table [id=4221fba1867e415f84267c33cb411b5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:28.543640 11748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f4d5f38cbe4c28bd106296d1276271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:28.548738 12178 tablet_bootstrap.cc:492] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: Bootstrap starting.
I20260811 03:01:28.550397 12185 tablet_bootstrap.cc:492] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Bootstrap starting.
I20260811 03:01:28.550946 12179 tablet_bootstrap.cc:492] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Bootstrap starting.
I20260811 03:01:28.551082 12178 tablet_bootstrap.cc:654] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:28.551898 12178 log.cc:824] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:28.552677 12185 tablet_bootstrap.cc:654] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:28.552932 12179 tablet_bootstrap.cc:654] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:28.553532 12185 log.cc:824] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:28.553701 12179 log.cc:824] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:28.554620 12178 tablet_bootstrap.cc:492] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: No bootstrap required, opened a new log
I20260811 03:01:28.554827 12178 ts_tablet_manager.cc:1397] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:01:28.555253 12185 tablet_bootstrap.cc:492] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: No bootstrap required, opened a new log
I20260811 03:01:28.555428 12185 ts_tablet_manager.cc:1397] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:01:28.555599 12179 tablet_bootstrap.cc:492] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: No bootstrap required, opened a new log
I20260811 03:01:28.555780 12179 ts_tablet_manager.cc:1397] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:01:28.558467 12185 raft_consensus.cc:348] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.558678 12185 raft_consensus.cc:374] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:28.558730 12185 raft_consensus.cc:729] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48219a9b4eac4275adf97f0713d0af50, State: Initialized, Role: FOLLOWER
I20260811 03:01:28.558626 12178 raft_consensus.cc:348] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.558878 12178 raft_consensus.cc:374] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:28.558934 12178 raft_consensus.cc:729] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7b1dd2c9e464904b6c18783ccb76818, State: Initialized, Role: FOLLOWER
I20260811 03:01:28.559145 12185 consensus_queue.cc:260] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.559355 12178 consensus_queue.cc:260] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.560146 12185 ts_tablet_manager.cc:1428] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:28.560386 12178 ts_tablet_manager.cc:1428] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 03:01:28.560431 11871 heartbeater.cc:503] Master 127.8.203.126:34627 was elected leader, sending a full tablet report...
I20260811 03:01:28.560695 11632 heartbeater.cc:503] Master 127.8.203.126:34627 was elected leader, sending a full tablet report...
I20260811 03:01:28.560129 12179 raft_consensus.cc:348] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.561334 12179 raft_consensus.cc:374] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:28.561395 12179 raft_consensus.cc:729] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750c3f3e035942d68e283bdec3092ef8, State: Initialized, Role: FOLLOWER
I20260811 03:01:28.561800 12179 consensus_queue.cc:260] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.562438 12152 heartbeater.cc:503] Master 127.8.203.126:34627 was elected leader, sending a full tablet report...
I20260811 03:01:28.562668 12179 ts_tablet_manager.cc:1428] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260811 03:01:28.595355 12195 raft_consensus.cc:482] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:28.595561 12195 raft_consensus.cc:504] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
W20260811 03:01:28.596236 11635 tablet.cc:2369] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:01:28.597024 12195 leader_election.cc:288] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7b1dd2c9e464904b6c18783ccb76818 (127.8.203.65:34473), 750c3f3e035942d68e283bdec3092ef8 (127.8.203.67:34111)
I20260811 03:01:28.601347 12177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.597471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:42721 (local address 127.8.203.67:34111)
0811 03:01:28.597575 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.597579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.597801 (+ 222us) server_negotiation.cc:407] Connection header received
0811 03:01:28.597894 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.597897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.597945 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.598023 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:28.598254 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.598909 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.600190 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.600769 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.600862 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.600902 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.601129 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.601198 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 03:01:28.601347 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.597320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:44567 (local address 127.8.203.65:34473)
0811 03:01:28.597446 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.597451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.597465 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:01:28.597518 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.597521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.597571 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.597649 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:28.597948 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.598691 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.600128 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.600711 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.600831 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.600882 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.601129 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.601198 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:01:28.601413 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.597151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34473 (local address 127.8.203.66:44567)
0811 03:01:28.597259 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.597275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.597385 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.597689 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.597692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.597712 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:28.597843 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.597851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.598798 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.598800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.600008 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.600020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.600841 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.600845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.601024 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.601041 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.601119 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.601225 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":72}
I20260811 03:01:28.601423 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.597271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:34111 (local address 127.8.203.66:42721)
0811 03:01:28.597707 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.597719 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:28.597809 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:28.598040 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:28.598042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:28.598061 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:28.598160 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.598165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.598997 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.599000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.600094 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:28.600102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.600863 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.600866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:28.600995 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.601032 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 03:01:28.601109 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 03:01:28.601218 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":46,"thread_start_us":137,"threads_started":1}
I20260811 03:01:28.602015 11540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f4d5f38cbe4c28bd106296d1276271" candidate_uuid: "48219a9b4eac4275adf97f0713d0af50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7b1dd2c9e464904b6c18783ccb76818" is_pre_election: true
I20260811 03:01:28.602025 12053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f4d5f38cbe4c28bd106296d1276271" candidate_uuid: "48219a9b4eac4275adf97f0713d0af50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750c3f3e035942d68e283bdec3092ef8" is_pre_election: true
I20260811 03:01:28.602288 11540 raft_consensus.cc:2454] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48219a9b4eac4275adf97f0713d0af50 in term 0.
I20260811 03:01:28.602300 12053 raft_consensus.cc:2454] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48219a9b4eac4275adf97f0713d0af50 in term 0.
I20260811 03:01:28.602766 11667 leader_election.cc:302] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 48219a9b4eac4275adf97f0713d0af50, 750c3f3e035942d68e283bdec3092ef8, e7b1dd2c9e464904b6c18783ccb76818; no voters:
I20260811 03:01:28.603010 12195 raft_consensus.cc:2781] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:01:28.603096 12195 raft_consensus.cc:482] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:01:28.603147 12195 raft_consensus.cc:3037] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:28.604116 12195 raft_consensus.cc:504] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.604413 12195 leader_election.cc:288] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [CANDIDATE]: Term 1 election: Requested vote from peers e7b1dd2c9e464904b6c18783ccb76818 (127.8.203.65:34473), 750c3f3e035942d68e283bdec3092ef8 (127.8.203.67:34111)
I20260811 03:01:28.604707 12053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f4d5f38cbe4c28bd106296d1276271" candidate_uuid: "48219a9b4eac4275adf97f0713d0af50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750c3f3e035942d68e283bdec3092ef8"
I20260811 03:01:28.604832 12053 raft_consensus.cc:3037] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:28.605406 11540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f4d5f38cbe4c28bd106296d1276271" candidate_uuid: "48219a9b4eac4275adf97f0713d0af50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7b1dd2c9e464904b6c18783ccb76818"
I20260811 03:01:28.605521 11540 raft_consensus.cc:3037] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:28.606534 11540 raft_consensus.cc:2454] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48219a9b4eac4275adf97f0713d0af50 in term 1.
I20260811 03:01:28.606822 11667 leader_election.cc:302] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48219a9b4eac4275adf97f0713d0af50, e7b1dd2c9e464904b6c18783ccb76818; no voters:
I20260811 03:01:28.606990 12195 raft_consensus.cc:2781] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:28.607180 12195 raft_consensus.cc:686] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 1 LEADER]: Becoming Leader. State: Replica: 48219a9b4eac4275adf97f0713d0af50, State: Running, Role: LEADER
I20260811 03:01:28.607405 12053 raft_consensus.cc:2454] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48219a9b4eac4275adf97f0713d0af50 in term 1.
I20260811 03:01:28.607509 12195 consensus_queue.cc:237] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } }
I20260811 03:01:28.610525 11274 catalog_manager.cc:5495] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48219a9b4eac4275adf97f0713d0af50 (127.8.203.66). New cstate: current_term: 1 leader_uuid: "48219a9b4eac4275adf97f0713d0af50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48219a9b4eac4275adf97f0713d0af50" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 } health_report { overall_health: UNKNOWN } } }
I20260811 03:01:28.637586 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.633824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.8.203.66:41883)
0811 03:01:28.633939 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:28.633943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:28.633986 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:01:28.634141 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:28.634144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:28.634198 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:28.634279 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:01:28.634520 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.635006 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.636207 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:28.636412 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:28.636477 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:28.636761 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:01:28.637174 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:01:28.637262 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:28.637397 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 03:01:28.637446 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:01:28.643967 12053 raft_consensus.cc:1261] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 1 FOLLOWER]: Refusing update from remote peer 48219a9b4eac4275adf97f0713d0af50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:28.643967 11540 raft_consensus.cc:1261] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 1 FOLLOWER]: Refusing update from remote peer 48219a9b4eac4275adf97f0713d0af50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:28.644694 12195 consensus_queue.cc:1035] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750c3f3e035942d68e283bdec3092ef8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 34111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:28.644881 12200 consensus_queue.cc:1035] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7b1dd2c9e464904b6c18783ccb76818" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 34473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:28.652873 12210 mvcc.cc:205] Tried to move back new op lower bound from 7317165214278324224 to 7317165214138867712. Current Snapshot: MvccSnapshot[applied={T|T < 7317165214278324224}]
I20260811 03:01:28.654932 12207 mvcc.cc:205] Tried to move back new op lower bound from 7317165214278324224 to 7317165214138867712. Current Snapshot: MvccSnapshot[applied={T|T < 7317165214278324224}]
I20260811 03:01:28.655499 12209 mvcc.cc:205] Tried to move back new op lower bound from 7317165214278324224 to 7317165214138867712. Current Snapshot: MvccSnapshot[applied={T|T < 7317165214278324224 or (T in {7317165214278324224})}]
I20260811 03:01:28.664652 11274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:28.664830 11274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:28.667639 11274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b19ebf72d60d4575a8e921d6bbafa890: Sending DeleteTablet for 3 replicas of tablet 89f4d5f38cbe4c28bd106296d1276271
I20260811 03:01:28.668385 11748 tablet_service.cc:1514] Processing DeleteTablet for tablet 89f4d5f38cbe4c28bd106296d1276271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:28 UTC) from {username='slave'} at 127.0.0.1:39186
I20260811 03:01:28.668845 12229 tablet_replica.cc:330] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: stopping tablet replica
I20260811 03:01:28.668792 12013 tablet_service.cc:1514] Processing DeleteTablet for tablet 89f4d5f38cbe4c28bd106296d1276271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:28 UTC) from {username='slave'} at 127.0.0.1:46910
I20260811 03:01:28.668862 11501 tablet_service.cc:1514] Processing DeleteTablet for tablet 89f4d5f38cbe4c28bd106296d1276271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:28 UTC) from {username='slave'} at 127.0.0.1:51224
I20260811 03:01:28.669126 12229 raft_consensus.cc:2229] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:01:28.669543 12229 raft_consensus.cc:2258] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:28.669914 12231 tablet_replica.cc:330] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818: stopping tablet replica
I20260811 03:01:28.670248 12231 raft_consensus.cc:2229] T 89f4d5f38cbe4c28bd106296d1276271 P e7b1dd2c9e464904b6c18783ccb76818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:28.670362 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11384
I20260811 03:01:28.671021 12229 ts_tablet_manager.cc:1905] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:28.673333 12229 ts_tablet_manager.cc:1918] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:28.673418 12229 log.cc:1196] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/89f4d5f38cbe4c28bd106296d1276271
I20260811 03:01:28.673733 12229 ts_tablet_manager.cc:1939] T 89f4d5f38cbe4c28bd106296d1276271 P 48219a9b4eac4275adf97f0713d0af50: Deleting consensus metadata
I20260811 03:01:28.678457 11254 catalog_manager.cc:4841] TS 48219a9b4eac4275adf97f0713d0af50 (127.8.203.66:41883): tablet 89f4d5f38cbe4c28bd106296d1276271 (table test-table [id=4221fba1867e415f84267c33cb411b5d]) successfully deleted
I20260811 03:01:28.687953 12230 tablet_replica.cc:330] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: stopping tablet replica
I20260811 03:01:28.688324 12230 raft_consensus.cc:2229] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:28.688704 12230 raft_consensus.cc:2258] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:28.690187 12230 ts_tablet_manager.cc:1905] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:28.693027 12230 ts_tablet_manager.cc:1918] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:28.693138 12230 log.cc:1196] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/89f4d5f38cbe4c28bd106296d1276271
I20260811 03:01:28.693491 12230 ts_tablet_manager.cc:1939] T 89f4d5f38cbe4c28bd106296d1276271 P 750c3f3e035942d68e283bdec3092ef8: Deleting consensus metadata
I20260811 03:01:28.694752 11255 catalog_manager.cc:4841] TS 750c3f3e035942d68e283bdec3092ef8 (127.8.203.67:34111): tablet 89f4d5f38cbe4c28bd106296d1276271 (table test-table [id=4221fba1867e415f84267c33cb411b5d]) successfully deleted
W20260811 03:01:28.696228 11247 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:34473 (error 108)
I20260811 03:01:28.696415 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11639
I20260811 03:01:28.714493 12168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:28.714050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:34473 (local address 127.0.0.1:51236)
0811 03:01:28.714289 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:28.714382 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:34473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260811 03:01:28.714718 11247 catalog_manager.cc:4584] TS e7b1dd2c9e464904b6c18783ccb76818 (127.8.203.65:34473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89f4d5f38cbe4c28bd106296d1276271: Network error: Client connection negotiation failed: client connection to 127.8.203.65:34473: connect: Connection refused (error 111)
I20260811 03:01:28.718806 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11876
I20260811 03:01:28.740562 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 11213
2026-08-11T03:01:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1078 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:28 dist-test-slave-gjw7 krb5kdc[12235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:28 dist-test-slave-gjw7 krb5kdc[12235](info): set up 2 sockets
Aug 11 03:01:28 dist-test-slave-gjw7 krb5kdc[12235](info): commencing operation
krb5kdc: starting...
W20260811 03:01:29.965066 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:01:30.081781 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36309
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:36309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:30.190922 12259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:30.191149 12259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:30.191186 12259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:30.195019 12259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:30.195066 12259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:30.195084 12259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:30.195104 12259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:30.195118 12259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:30.200093 12259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:36309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:30.200999 12259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:30.201954 12259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:30.209966 12280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:30.209991 12274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:30.209975 12273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:30.210021 12259 server_base.cc:1034] running on GCE node
I20260811 03:01:30.210819 12259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:30.211861 12259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:30.213024 12259 hybrid_clock.cc:648] HybridClock initialized: now 1786417290213008 us; error 27 us; skew 500 ppm
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:30.215633 12259 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:01:30.216509 12259 webserver.cc:460] Webserver started at http://127.8.203.126:40781/ using document root <none> and password file <none>
I20260811 03:01:30.217060 12259 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:30.217127 12259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:30.217367 12259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:30.219456 12259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "f48a8b70e1a14541ad6f147201f4bffb"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "14c59724cf978b0bcad5f52d1bc29b29"
server_key_iv: "380b549f774ede476f496d9c1827cd96"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.220027 12259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "f48a8b70e1a14541ad6f147201f4bffb"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "14c59724cf978b0bcad5f52d1bc29b29"
server_key_iv: "380b549f774ede476f496d9c1827cd96"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.223619 12259 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 03:01:30.225909 12291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.227067 12259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:30.227187 12259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "f48a8b70e1a14541ad6f147201f4bffb"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "14c59724cf978b0bcad5f52d1bc29b29"
server_key_iv: "380b549f774ede476f496d9c1827cd96"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.227286 12259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:30.247704 12259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:30.254554 12259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:30.254724 12259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:30.261305 12259 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:36309
I20260811 03:01:30.261380 12383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:36309 every 8 connection(s)
I20260811 03:01:30.262243 12259 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:30.264994 12384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:30.267369 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 12259
I20260811 03:01:30.267490 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:30.267781 9005 external_mini_cluster.cc:1428] Setting key 3eefbd0ee5bda121e0ffdf0731e8b103
I20260811 03:01:30.270641 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb: Bootstrap starting.
I20260811 03:01:30.273092 12384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:30.273957 12384 log.cc:824] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:30.276098 12384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb: No bootstrap required, opened a new log
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:30.279067 12384 raft_consensus.cc:348] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48a8b70e1a14541ad6f147201f4bffb" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36309 } }
I20260811 03:01:30.279279 12384 raft_consensus.cc:374] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:30.279349 12384 raft_consensus.cc:729] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f48a8b70e1a14541ad6f147201f4bffb, State: Initialized, Role: FOLLOWER
I20260811 03:01:30.279759 12384 consensus_queue.cc:260] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48a8b70e1a14541ad6f147201f4bffb" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36309 } }
I20260811 03:01:30.279899 12384 raft_consensus.cc:388] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:30.279964 12384 raft_consensus.cc:482] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:30.280057 12384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:30.280829 12384 raft_consensus.cc:504] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48a8b70e1a14541ad6f147201f4bffb" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36309 } }
I20260811 03:01:30.281155 12384 leader_election.cc:302] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f48a8b70e1a14541ad6f147201f4bffb; no voters:
I20260811 03:01:30.281448 12384 leader_election.cc:288] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:30.281540 12392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:30.281738 12392 raft_consensus.cc:686] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [term 1 LEADER]: Becoming Leader. State: Replica: f48a8b70e1a14541ad6f147201f4bffb, State: Running, Role: LEADER
I20260811 03:01:30.281865 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.269020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33518 (local address 127.8.203.126:36309)
0811 03:01:30.269420 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.269436 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.269471 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:01:30.270085 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.270103 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.270426 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.270692 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:30.271536 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.272519 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.273707 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.274209 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.274369 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.276865 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:30.276870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:30.276883 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:30.276909 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:30.278909 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.279310 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.279316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.279322 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.279458 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.279722 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.279725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.279727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.280185 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:30.280378 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.280665 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.280946 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":105,"threads_started":1}
I20260811 03:01:30.282094 12392 consensus_queue.cc:237] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48a8b70e1a14541ad6f147201f4bffb" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36309 } }
I20260811 03:01:30.282593 12384 sys_catalog.cc:564] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:30.283635 12393 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f48a8b70e1a14541ad6f147201f4bffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48a8b70e1a14541ad6f147201f4bffb" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36309 } } }
I20260811 03:01:30.283753 12393 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:30.284089 12395 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f48a8b70e1a14541ad6f147201f4bffb. Latest consensus state: current_term: 1 leader_uuid: "f48a8b70e1a14541ad6f147201f4bffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48a8b70e1a14541ad6f147201f4bffb" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36309 } } }
I20260811 03:01:30.284166 12395 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:30.285156 12406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:30.288012 12406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:30.292793 12406 catalog_manager.cc:1347] Generated new cluster ID: 24be718e48e54fc5a66f9c198cacd391
I20260811 03:01:30.292865 12406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:30.304792 12406 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:30.305768 12406 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:30.312779 12406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb: Generated new TSK 0
I20260811 03:01:30.313447 12406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 03:01:30.341416 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:36309
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:30.452030 12426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:30.452262 12426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:30.452301 12426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:30.456192 12426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:30.456243 12426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:30.456297 12426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:01:30.461247 12426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:36309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:30.462182 12426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:30.463155 12426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:30.472391 12441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:30.472533 12426 server_base.cc:1034] running on GCE node
W20260811 03:01:30.472435 12436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:30.472391 12438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:30.473002 12426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:30.473670 12426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:30.474829 12426 hybrid_clock.cc:648] HybridClock initialized: now 1786417290474809 us; error 36 us; skew 500 ppm
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:30.477396 12426 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:01:30.478209 12426 webserver.cc:460] Webserver started at http://127.8.203.65:37337/ using document root <none> and password file <none>
I20260811 03:01:30.478726 12426 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:30.478790 12426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:30.478981 12426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:30.480945 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "b8f4d5ffedd540b0ba6a584471c8a358"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "70d62fcd8012fcfe52efcbf2565ea475"
server_key_iv: "ac2ee6e0945b2b501006dd240c377fbd"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.481515 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "b8f4d5ffedd540b0ba6a584471c8a358"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "70d62fcd8012fcfe52efcbf2565ea475"
server_key_iv: "ac2ee6e0945b2b501006dd240c377fbd"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.485184 12426 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:01:30.487464 12460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.488592 12426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:30.488706 12426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "b8f4d5ffedd540b0ba6a584471c8a358"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "70d62fcd8012fcfe52efcbf2565ea475"
server_key_iv: "ac2ee6e0945b2b501006dd240c377fbd"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.488794 12426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:30.509795 12426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:30.512447 12426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:30.512624 12426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:30.513197 12426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:30.514048 12426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:30.514097 12426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.514150 12426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:30.514178 12426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.522423 12426 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:41467
I20260811 03:01:30.522446 12692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:41467 every 8 connection(s)
I20260811 03:01:30.523268 12426 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:01:30.527762 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 12426
I20260811 03:01:30.527915 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:01:30.528240 9005 external_mini_cluster.cc:1428] Setting key 5afc05e7aa38d6d478c5e1d87c748e5f
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:30.536746 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.525011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:38821 (local address 127.8.203.126:36309)
0811 03:01:30.525189 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.525194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.526045 (+ 851us) server_negotiation.cc:407] Connection header received
0811 03:01:30.527296 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.527299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.527351 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.527422 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:30.528850 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.529498 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.530675 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.530970 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.531073 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.533482 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:30.533485 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:30.533487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:30.533510 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:30.534846 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.535310 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.535314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.535315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.535397 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.535706 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.535710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.535711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.535883 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:30.535978 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.536458 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.536585 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 03:01:30.537593 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.525451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:36309 (local address 127.8.203.65:38821)
0811 03:01:30.525899 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:30.525942 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:30.527110 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:30.527593 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:30.527602 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:30.528079 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:30.528717 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.528735 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.529616 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.529620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.530559 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.530571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.531096 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.531098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.531262 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.531847 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:30.531863 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:30.533329 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:30.534951 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:30.534957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:30.534971 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:30.535218 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:30.535480 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:30.535483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:30.535486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:30.535621 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:30.535995 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:30.536002 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:30.536232 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 03:01:30.536400 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 03:01:30.536711 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
I20260811 03:01:30.538772 12693 heartbeater.cc:347] Connected to a master server at 127.8.203.126:36309
I20260811 03:01:30.538978 12693 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:30.539489 12693 heartbeater.cc:511] Master 127.8.203.126:36309 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
I20260811 03:01:30.540999 12321 ts_manager.cc:194] Registered new tserver with Master: b8f4d5ffedd540b0ba6a584471c8a358 (127.8.203.65:41467)
I20260811 03:01:30.542140 12321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:38821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 03:01:30.555953 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:36309
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:30.665134 12699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:30.665392 12699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:30.665432 12699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:30.669279 12699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:30.669335 12699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:30.669389 12699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:01:30.674392 12699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:36309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:30.675307 12699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:30.676229 12699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:30.685276 12714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:30.685371 12712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:30.685380 12710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:30.685493 12699 server_base.cc:1034] running on GCE node
I20260811 03:01:30.685930 12699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:30.686509 12699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:30.687654 12699 hybrid_clock.cc:648] HybridClock initialized: now 1786417290687631 us; error 41 us; skew 500 ppm
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:30.690128 12699 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:01:30.690999 12699 webserver.cc:460] Webserver started at http://127.8.203.66:39565/ using document root <none> and password file <none>
I20260811 03:01:30.691529 12699 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:30.691587 12699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:30.691776 12699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:30.693790 12699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "8278182995cd471382465b6a25a706ff"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "b6717e87234bd15af69277bcc9f39877"
server_key_iv: "b810375100b55e3eff135433cd34f18b"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.694316 12699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "8278182995cd471382465b6a25a706ff"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "b6717e87234bd15af69277bcc9f39877"
server_key_iv: "b810375100b55e3eff135433cd34f18b"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.697814 12699 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:01:30.700037 12727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.701148 12699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:30.701241 12699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "8278182995cd471382465b6a25a706ff"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "b6717e87234bd15af69277bcc9f39877"
server_key_iv: "b810375100b55e3eff135433cd34f18b"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.701363 12699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:30.721119 12699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:30.723742 12699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:30.723906 12699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:30.724466 12699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:30.725261 12699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:30.725356 12699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.725441 12699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:30.725471 12699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.733930 12699 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:43841
I20260811 03:01:30.733958 12928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:43841 every 8 connection(s)
I20260811 03:01:30.734790 12699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:01:30.741953 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 12699
I20260811 03:01:30.742077 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:01:30.742414 9005 external_mini_cluster.cc:1428] Setting key 9c5b54ad0961fb70dcb85d96e3d9b25d
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:30.748175 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.736540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:46465 (local address 127.8.203.126:36309)
0811 03:01:30.736678 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.736682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.737520 (+ 838us) server_negotiation.cc:407] Connection header received
0811 03:01:30.738809 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.738813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.738883 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.738954 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:30.740120 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.740653 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.741911 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.742187 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.742273 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.744763 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:30.744767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:30.744770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:30.744795 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:30.746201 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.746761 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.746763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.746764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.746838 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.747170 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.747171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.747173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.747328 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:30.747403 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.747912 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.748030 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:01:30.748883 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.736913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:36309 (local address 127.8.203.66:46465)
0811 03:01:30.737384 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:30.737429 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:30.738603 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:30.739143 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:30.739150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:30.739505 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:30.739976 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.739991 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.740773 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.740776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.741796 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.741808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.742365 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.742371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.742579 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.743159 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:30.743177 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:30.744605 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:30.746314 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:30.746320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:30.746335 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:30.746675 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:30.746917 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:30.746920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:30.746922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:30.747094 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:30.747418 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:30.747423 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:30.747660 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 03:01:30.747815 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 03:01:30.748062 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":116,"threads_started":1}
I20260811 03:01:30.749980 12931 heartbeater.cc:347] Connected to a master server at 127.8.203.126:36309
I20260811 03:01:30.750238 12931 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:30.750818 12931 heartbeater.cc:511] Master 127.8.203.126:36309 requested a full tablet report, sending...
I20260811 03:01:30.751961 12321 ts_manager.cc:194] Registered new tserver with Master: 8278182995cd471382465b6a25a706ff (127.8.203.66:43841)
I20260811 03:01:30.752708 12321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:46465
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 03:01:30.769896 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:36309
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:30.879079 12940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:30.879460 12940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:30.879572 12940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:30.885149 12940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:30.885219 12940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:30.885344 12940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:01:30.892694 12940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:36309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:30.894023 12940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:30.895325 12940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:30.906699 12952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:30.908123 12947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:30.910681 12946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:30.910984 12940 server_base.cc:1034] running on GCE node
I20260811 03:01:30.911455 12940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:30.912099 12940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:30.913268 12940 hybrid_clock.cc:648] HybridClock initialized: now 1786417290913233 us; error 45 us; skew 500 ppm
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:30.916033 12940 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:01:30.916886 12940 webserver.cc:460] Webserver started at http://127.8.203.67:35843/ using document root <none> and password file <none>
I20260811 03:01:30.917493 12940 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:30.917559 12940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:30.917768 12940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:30.919837 12940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "77a16a1bef2c4678b975ee0d555bad9c"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "80145a692d4116f9a4365922cdd9f5fa"
server_key_iv: "5b24d4bfc001cfc19fde6689a9b30b63"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.920395 12940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "77a16a1bef2c4678b975ee0d555bad9c"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "80145a692d4116f9a4365922cdd9f5fa"
server_key_iv: "5b24d4bfc001cfc19fde6689a9b30b63"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.924047 12940 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 03:01:30.926460 12971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.927583 12940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:30.927681 12940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "77a16a1bef2c4678b975ee0d555bad9c"
format_stamp: "Formatted at 2026-08-11 03:01:30 on dist-test-slave-gjw7"
server_key: "80145a692d4116f9a4365922cdd9f5fa"
server_key_iv: "5b24d4bfc001cfc19fde6689a9b30b63"
server_key_version: "encryptionkey@0"
I20260811 03:01:30.927778 12940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:30.937669 12940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:30.940507 12940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:30.940665 12940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:30.941195 12940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:30.942009 12940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:30.942054 12940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.942106 12940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:30.942137 12940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:30.949898 12940 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:35675
I20260811 03:01:30.949921 13167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:35675 every 8 connection(s)
I20260811 03:01:30.950680 12940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:01:30.955435 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 12940
I20260811 03:01:30.955538 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:01:30.955817 9005 external_mini_cluster.cc:1428] Setting key aa3e7043076b3cd38e1c7308e7f3dfd0
Aug 11 03:01:30 dist-test-slave-gjw7 krb5kdc[12235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:30.963019 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.952286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:59723 (local address 127.8.203.126:36309)
0811 03:01:30.952399 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.952403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.953165 (+ 762us) server_negotiation.cc:407] Connection header received
0811 03:01:30.954393 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.954395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.954444 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.954514 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:30.955604 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.956305 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.957784 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.958069 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.958147 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.960248 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:30.960252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:30.960254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:30.960277 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:30.961537 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.961922 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.961925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.961926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.961990 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.962219 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.962221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.962222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.962372 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:30.962439 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.962813 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.962907 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 03:01:30.963830 13170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.952599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:36309 (local address 127.8.203.67:59723)
0811 03:01:30.953017 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:30.953057 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:30.954204 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:30.954659 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:30.954666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:30.955006 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:30.955453 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.955465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.956418 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.956421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.957673 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.957684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.958190 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.958193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.958327 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.958777 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:30.958792 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:30.960110 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:30.961613 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:30.961616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:30.961626 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:30.961852 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:30.962047 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:30.962049 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:30.962050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:30.962160 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:30.962437 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:30.962441 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:30.962643 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 03:01:30.962783 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 03:01:30.963018 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260811 03:01:30.964840 13168 heartbeater.cc:347] Connected to a master server at 127.8.203.126:36309
I20260811 03:01:30.965045 13168 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:30.965512 13168 heartbeater.cc:511] Master 127.8.203.126:36309 requested a full tablet report, sending...
I20260811 03:01:30.966432 12321 ts_manager.cc:194] Registered new tserver with Master: 77a16a1bef2c4678b975ee0d555bad9c (127.8.203.67:35675)
I20260811 03:01:30.966940 12321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:59723
I20260811 03:01:30.969424 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:01:30.977389 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.971157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33524 (local address 127.8.203.126:36309)
0811 03:01:30.971283 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.971287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.971414 (+ 127us) server_negotiation.cc:407] Connection header received
0811 03:01:30.971516 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.971518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.971552 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.971601 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:30.972146 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.972604 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.973336 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.973542 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.973609 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.974855 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:30.974858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:30.974859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:30.974886 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:30.976059 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.976441 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.976443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.976445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.976508 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:30.976720 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:30.976721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:30.976722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:30.976894 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:30.976997 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.977160 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.977226 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 03:01:30.979772 12321 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:01:30.982757 12321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:01:30.996882 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.990763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.8.203.65:41467)
0811 03:01:30.991080 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.991096 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.991909 (+ 813us) server_negotiation.cc:407] Connection header received
0811 03:01:30.991991 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.991996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.992208 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.992329 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:30.992591 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.993748 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.995488 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.996180 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.996263 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.996370 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.996633 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.996738 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":136,"threads_started":1}
I20260811 03:01:30.996922 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.991374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44732 (local address 127.8.203.66:43841)
0811 03:01:30.992880 (+ 1506us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.992893 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.992915 (+ 22us) server_negotiation.cc:407] Connection header received
0811 03:01:30.992984 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.992989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.993153 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.993570 (+ 417us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:30.993718 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.994694 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.995775 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.996370 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.996470 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.996602 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.996714 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.996792 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":143,"threads_started":1}
I20260811 03:01:30.996923 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.990757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43841 (local address 127.0.0.1:44732)
0811 03:01:30.991606 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:30.991651 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:30.991761 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:30.993328 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:30.993331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:30.993350 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:30.993434 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.993439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.994849 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.994853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.995659 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.995669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.996488 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.996492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.996582 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.996605 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 03:01:30.996678 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 03:01:30.996791 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":91,"threads_started":1}
I20260811 03:01:30.997697 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.990635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:41467 (local address 127.0.0.1:54506)
0811 03:01:30.991688 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:30.991703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:30.991798 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:30.992342 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:30.992347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:30.992368 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:30.992475 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.992479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.993890 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.993893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.995359 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.995371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.996297 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.996301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.996416 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.996462 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 03:01:30.997240 (+ 778us) client_negotiation.cc:240] Negotiation successful
0811 03:01:30.997573 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":29,"thread_start_us":58,"threads_started":1}
I20260811 03:01:30.998183 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.991290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:35675 (local address 127.0.0.1:34060)
0811 03:01:30.992531 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:30.992543 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:30.992635 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:30.992981 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:30.992984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:30.993000 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:30.993078 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.993084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.994196 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.994199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.995362 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:30.995376 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.997722 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.997726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:30.997910 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.997934 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:01:30.998000 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 03:01:30.998065 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1187,"mutex_wait_us":20,"thread_start_us":90,"threads_started":1}
I20260811 03:01:30.998541 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:30.991413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.8.203.67:35675)
0811 03:01:30.992031 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:30.992040 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:30.992614 (+ 574us) server_negotiation.cc:407] Connection header received
0811 03:01:30.992725 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:30.992729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:30.992864 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:30.992981 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:30.993161 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.994052 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.996420 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:30.997430 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:30.997529 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:30.997662 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:30.998341 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 03:01:30.998438 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":132,"threads_started":1}
I20260811 03:01:30.999812 12823 tablet_service.cc:1467] Processing CreateTablet for tablet eb47a7a5465246b1aadac9c5d827a0bb (DEFAULT_TABLE table=test-table [id=a881f6e17cf749d29618f16ad28162d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:31.000977 12823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb47a7a5465246b1aadac9c5d827a0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:31.000844 13048 tablet_service.cc:1467] Processing CreateTablet for tablet eb47a7a5465246b1aadac9c5d827a0bb (DEFAULT_TABLE table=test-table [id=a881f6e17cf749d29618f16ad28162d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:31.001196 12554 tablet_service.cc:1467] Processing CreateTablet for tablet eb47a7a5465246b1aadac9c5d827a0bb (DEFAULT_TABLE table=test-table [id=a881f6e17cf749d29618f16ad28162d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:31.001852 13048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb47a7a5465246b1aadac9c5d827a0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:31.002170 12554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb47a7a5465246b1aadac9c5d827a0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:31.007876 13208 tablet_bootstrap.cc:492] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Bootstrap starting.
I20260811 03:01:31.009284 13215 tablet_bootstrap.cc:492] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: Bootstrap starting.
I20260811 03:01:31.009352 13209 tablet_bootstrap.cc:492] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: Bootstrap starting.
I20260811 03:01:31.010211 13208 tablet_bootstrap.cc:654] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:31.010963 13215 tablet_bootstrap.cc:654] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:31.011473 13208 log.cc:824] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:31.011771 13209 tablet_bootstrap.cc:654] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:31.012051 13215 log.cc:824] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:31.012557 13209 log.cc:824] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:31.013835 13208 tablet_bootstrap.cc:492] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: No bootstrap required, opened a new log
I20260811 03:01:31.013988 13215 tablet_bootstrap.cc:492] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: No bootstrap required, opened a new log
I20260811 03:01:31.014034 13208 ts_tablet_manager.cc:1397] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 03:01:31.014164 13215 ts_tablet_manager.cc:1397] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:01:31.014453 13209 tablet_bootstrap.cc:492] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: No bootstrap required, opened a new log
I20260811 03:01:31.014623 13209 ts_tablet_manager.cc:1397] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:01:31.016923 13215 raft_consensus.cc:348] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.017134 13215 raft_consensus.cc:374] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:31.017196 13215 raft_consensus.cc:729] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f4d5ffedd540b0ba6a584471c8a358, State: Initialized, Role: FOLLOWER
I20260811 03:01:31.017632 13215 consensus_queue.cc:260] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.017817 13208 raft_consensus.cc:348] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.018078 13208 raft_consensus.cc:374] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:31.018136 13208 raft_consensus.cc:729] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a16a1bef2c4678b975ee0d555bad9c, State: Initialized, Role: FOLLOWER
I20260811 03:01:31.018174 13209 raft_consensus.cc:348] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.018361 12693 heartbeater.cc:503] Master 127.8.203.126:36309 was elected leader, sending a full tablet report...
I20260811 03:01:31.018407 13209 raft_consensus.cc:374] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:31.018464 13209 raft_consensus.cc:729] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8278182995cd471382465b6a25a706ff, State: Initialized, Role: FOLLOWER
I20260811 03:01:31.018560 13208 consensus_queue.cc:260] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.018836 13209 consensus_queue.cc:260] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.019461 13208 ts_tablet_manager.cc:1428] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:31.019606 13215 ts_tablet_manager.cc:1428] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:31.019659 13209 ts_tablet_manager.cc:1428] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 03:01:31.019848 13168 heartbeater.cc:503] Master 127.8.203.126:36309 was elected leader, sending a full tablet report...
I20260811 03:01:31.019938 12931 heartbeater.cc:503] Master 127.8.203.126:36309 was elected leader, sending a full tablet report...
W20260811 03:01:31.024297 12694 tablet.cc:2369] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:01:31.040869 13221 raft_consensus.cc:482] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:31.041003 13221 raft_consensus.cc:504] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.041927 13221 leader_election.cc:288] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8f4d5ffedd540b0ba6a584471c8a358 (127.8.203.65:41467), 8278182995cd471382465b6a25a706ff (127.8.203.66:43841)
I20260811 03:01:31.045171 13170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.042106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:41467 (local address 127.8.203.67:56313)
0811 03:01:31.042268 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:31.042283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:31.042390 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:31.042601 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:31.042604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:31.042621 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:31.042727 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.042731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.043406 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.043409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.044182 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.044191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.044782 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.044786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.044890 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.044909 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 03:01:31.044975 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 03:01:31.045047 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":2}
I20260811 03:01:31.045171 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.042180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56313 (local address 127.8.203.65:41467)
0811 03:01:31.042289 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.042293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.042389 (+ 96us) server_negotiation.cc:407] Connection header received
0811 03:01:31.042475 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.042478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.042520 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.042583 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:31.042819 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.043306 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.044298 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.044679 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.044779 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.044829 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:31.044997 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 03:01:31.045060 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:01:31.045434 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.042248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:56971 (local address 127.8.203.66:43841)
0811 03:01:31.042348 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.042352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.042554 (+ 202us) server_negotiation.cc:407] Connection header received
0811 03:01:31.042661 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.042663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.042702 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.042761 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:31.043013 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.043472 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.044506 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.044907 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.045016 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.045063 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:31.045232 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 03:01:31.045326 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:01:31.045493 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.042155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43841 (local address 127.8.203.67:56971)
0811 03:01:31.042461 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:31.042472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:31.042564 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:31.042783 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:31.042786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:31.042801 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:31.042909 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.042914 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.043575 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.043578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.044377 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.044386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.045021 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.045024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.045122 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.045140 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 03:01:31.045203 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 03:01:31.045360 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":32,"thread_start_us":103,"threads_started":1}
I20260811 03:01:31.045784 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb47a7a5465246b1aadac9c5d827a0bb" candidate_uuid: "77a16a1bef2c4678b975ee0d555bad9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" is_pre_election: true
I20260811 03:01:31.046018 12857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb47a7a5465246b1aadac9c5d827a0bb" candidate_uuid: "77a16a1bef2c4678b975ee0d555bad9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8278182995cd471382465b6a25a706ff" is_pre_election: true
I20260811 03:01:31.046056 12594 raft_consensus.cc:2454] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77a16a1bef2c4678b975ee0d555bad9c in term 0.
I20260811 03:01:31.046250 12857 raft_consensus.cc:2454] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77a16a1bef2c4678b975ee0d555bad9c in term 0.
I20260811 03:01:31.046525 12979 leader_election.cc:302] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77a16a1bef2c4678b975ee0d555bad9c, b8f4d5ffedd540b0ba6a584471c8a358; no voters:
I20260811 03:01:31.046772 13221 raft_consensus.cc:2781] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:01:31.046846 13221 raft_consensus.cc:482] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:01:31.046897 13221 raft_consensus.cc:3037] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:31.047820 13221 raft_consensus.cc:504] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.048117 13221 leader_election.cc:288] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [CANDIDATE]: Term 1 election: Requested vote from peers b8f4d5ffedd540b0ba6a584471c8a358 (127.8.203.65:41467), 8278182995cd471382465b6a25a706ff (127.8.203.66:43841)
I20260811 03:01:31.048424 12594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb47a7a5465246b1aadac9c5d827a0bb" candidate_uuid: "77a16a1bef2c4678b975ee0d555bad9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f4d5ffedd540b0ba6a584471c8a358"
I20260811 03:01:31.048445 12857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb47a7a5465246b1aadac9c5d827a0bb" candidate_uuid: "77a16a1bef2c4678b975ee0d555bad9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8278182995cd471382465b6a25a706ff"
I20260811 03:01:31.048540 12594 raft_consensus.cc:3037] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:31.048540 12857 raft_consensus.cc:3037] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:31.049749 12594 raft_consensus.cc:2454] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77a16a1bef2c4678b975ee0d555bad9c in term 1.
I20260811 03:01:31.049749 12857 raft_consensus.cc:2454] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77a16a1bef2c4678b975ee0d555bad9c in term 1.
I20260811 03:01:31.050028 12979 leader_election.cc:302] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77a16a1bef2c4678b975ee0d555bad9c, b8f4d5ffedd540b0ba6a584471c8a358; no voters:
I20260811 03:01:31.050186 13221 raft_consensus.cc:2781] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:31.050411 13221 raft_consensus.cc:686] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 1 LEADER]: Becoming Leader. State: Replica: 77a16a1bef2c4678b975ee0d555bad9c, State: Running, Role: LEADER
I20260811 03:01:31.050715 13221 consensus_queue.cc:237] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } }
I20260811 03:01:31.053454 12318 catalog_manager.cc:5495] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 77a16a1bef2c4678b975ee0d555bad9c (127.8.203.67). New cstate: current_term: 1 leader_uuid: "77a16a1bef2c4678b975ee0d555bad9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77a16a1bef2c4678b975ee0d555bad9c" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 35675 } health_report { overall_health: HEALTHY } } }
I20260811 03:01:31.073222 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.069885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.8.203.67:35675)
0811 03:01:31.069986 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.069990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.070002 (+ 12us) server_negotiation.cc:407] Connection header received
0811 03:01:31.070139 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.070141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.070190 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.070270 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:01:31.070482 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.070987 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.071804 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.072013 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.072099 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.072425 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:01:31.072803 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:01:31.072935 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:31.073047 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 03:01:31.073086 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 03:01:31.078605 12857 raft_consensus.cc:1261] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 1 FOLLOWER]: Refusing update from remote peer 77a16a1bef2c4678b975ee0d555bad9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:31.078605 12594 raft_consensus.cc:1261] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 1 FOLLOWER]: Refusing update from remote peer 77a16a1bef2c4678b975ee0d555bad9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:31.079245 13225 consensus_queue.cc:1035] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8f4d5ffedd540b0ba6a584471c8a358" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:31.079406 13221 consensus_queue.cc:1035] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8278182995cd471382465b6a25a706ff" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 43841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:31.093649 12318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:31.093818 12318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33524:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:31.096174 12318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f48a8b70e1a14541ad6f147201f4bffb: Sending DeleteTablet for 3 replicas of tablet eb47a7a5465246b1aadac9c5d827a0bb
I20260811 03:01:31.096915 12823 tablet_service.cc:1514] Processing DeleteTablet for tablet eb47a7a5465246b1aadac9c5d827a0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:31 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:44732
I20260811 03:01:31.096915 12554 tablet_service.cc:1514] Processing DeleteTablet for tablet eb47a7a5465246b1aadac9c5d827a0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:31 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:54506
I20260811 03:01:31.097040 13048 tablet_service.cc:1514] Processing DeleteTablet for tablet eb47a7a5465246b1aadac9c5d827a0bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:31 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:34060
I20260811 03:01:31.097383 13243 tablet_replica.cc:330] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: stopping tablet replica
I20260811 03:01:31.097451 13246 tablet_replica.cc:330] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: stopping tablet replica
I20260811 03:01:31.097627 13245 tablet_replica.cc:330] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358: stopping tablet replica
I20260811 03:01:31.097746 13246 raft_consensus.cc:2229] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:01:31.097836 13243 raft_consensus.cc:2229] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:31.097872 13245 raft_consensus.cc:2229] T eb47a7a5465246b1aadac9c5d827a0bb P b8f4d5ffedd540b0ba6a584471c8a358 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:31.098030 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 12426
I20260811 03:01:31.098050 13246 raft_consensus.cc:2258] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:31.099509 13246 ts_tablet_manager.cc:1905] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:31.101601 13246 ts_tablet_manager.cc:1918] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:31.101671 13246 log.cc:1196] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/eb47a7a5465246b1aadac9c5d827a0bb
I20260811 03:01:31.101934 13246 ts_tablet_manager.cc:1939] T eb47a7a5465246b1aadac9c5d827a0bb P 77a16a1bef2c4678b975ee0d555bad9c: Deleting consensus metadata
I20260811 03:01:31.104655 12298 catalog_manager.cc:4841] TS 77a16a1bef2c4678b975ee0d555bad9c (127.8.203.67:35675): tablet eb47a7a5465246b1aadac9c5d827a0bb (table test-table [id=a881f6e17cf749d29618f16ad28162d6]) successfully deleted
I20260811 03:01:31.109516 13243 raft_consensus.cc:2258] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:31.110873 13243 ts_tablet_manager.cc:1905] T eb47a7a5465246b1aadac9c5d827a0bb P 8278182995cd471382465b6a25a706ff: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 03:01:31.111060 12298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:41467 (error 108)
I20260811 03:01:31.111968 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.111715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:41467 (local address 127.0.0.1:54510)
0811 03:01:31.111826 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:31.111891 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:41467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 03:01:31.112108 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 12699
W20260811 03:01:31.112249 12298 catalog_manager.cc:4584] TS b8f4d5ffedd540b0ba6a584471c8a358 (127.8.203.65:41467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb47a7a5465246b1aadac9c5d827a0bb: Network error: Client connection negotiation failed: client connection to 127.8.203.65:41467: connect: Connection refused (error 111)
I20260811 03:01:31.124539 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 12940
I20260811 03:01:31.127576 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.124619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:43841 (local address 127.0.0.1:44742)
0811 03:01:31.127430 (+ 2811us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:31.127491 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.66:43841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2719}
I20260811 03:01:31.136512 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 12259
2026-08-11T03:01:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2401 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T03:01:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:31Z Disabled control of system clock
I20260811 03:01:31.178016 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43683
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:01:31.287031 13253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:31.287278 13253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:31.287329 13253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:31.291225 13253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:31.291275 13253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:31.291294 13253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:31.291316 13253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:31.291344 13253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:31.295939 13253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:31.296905 13253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:31.298276 13253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:31.306129 13264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:31.306129 13267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:31.306211 13263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:31.306392 13253 server_base.cc:1034] running on GCE node
I20260811 03:01:31.306921 13253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:31.307920 13253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:31.309069 13253 hybrid_clock.cc:648] HybridClock initialized: now 1786417291309054 us; error 27 us; skew 500 ppm
I20260811 03:01:31.310781 13253 webserver.cc:460] Webserver started at http://127.8.203.126:34673/ using document root <none> and password file <none>
I20260811 03:01:31.311302 13253 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:31.311362 13253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:31.311573 13253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:31.313571 13253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "88f031632c9045ffb3686ee382c62823"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "1c9775b15de33023cc8f6b4183143780"
server_key_iv: "da5e9411f69eef08b6a17d9307d2f7c2"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.314100 13253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "88f031632c9045ffb3686ee382c62823"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "1c9775b15de33023cc8f6b4183143780"
server_key_iv: "da5e9411f69eef08b6a17d9307d2f7c2"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.317548 13253 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:01:31.319759 13278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.320883 13253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:31.320983 13253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "88f031632c9045ffb3686ee382c62823"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "1c9775b15de33023cc8f6b4183143780"
server_key_iv: "da5e9411f69eef08b6a17d9307d2f7c2"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.321076 13253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:31.358280 13253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:31.359031 13253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:31.359244 13253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:31.366683 13253 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:43683
I20260811 03:01:31.366683 13385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:43683 every 8 connection(s)
I20260811 03:01:31.367718 13253 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:31.371081 13387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:31.373202 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 13253
I20260811 03:01:31.373390 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:31.373704 9005 external_mini_cluster.cc:1428] Setting key 36bd5f9b77c91a09e6a5416ba93e1daa
I20260811 03:01:31.377097 13387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823: Bootstrap starting.
I20260811 03:01:31.379534 13387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:31.380321 13387 log.cc:824] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:31.382535 13387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823: No bootstrap required, opened a new log
I20260811 03:01:31.382957 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.375092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.8.203.126:43683)
0811 03:01:31.375715 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.375737 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.375773 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:01:31.376414 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.376430 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.376817 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.377097 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:31.377512 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.378447 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.379580 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.380058 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.380167 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.380972 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:31.380977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:31.380989 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:31.381484 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:31.381554 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:31.381580 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:31.381670 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:31.381792 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 03:01:31.382061 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":119,"threads_started":1}
I20260811 03:01:31.386300 13387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f031632c9045ffb3686ee382c62823" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43683 } }
I20260811 03:01:31.386507 13387 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:31.386546 13387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88f031632c9045ffb3686ee382c62823, State: Initialized, Role: FOLLOWER
I20260811 03:01:31.386888 13387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f031632c9045ffb3686ee382c62823" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43683 } }
I20260811 03:01:31.386996 13387 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:31.387038 13387 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:31.387104 13387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:31.387871 13387 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f031632c9045ffb3686ee382c62823" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43683 } }
I20260811 03:01:31.388157 13387 leader_election.cc:302] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88f031632c9045ffb3686ee382c62823; no voters:
I20260811 03:01:31.388404 13387 leader_election.cc:288] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:31.388494 13394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:31.388746 13394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [term 1 LEADER]: Becoming Leader. State: Replica: 88f031632c9045ffb3686ee382c62823, State: Running, Role: LEADER
I20260811 03:01:31.389053 13394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f031632c9045ffb3686ee382c62823" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43683 } }
I20260811 03:01:31.389568 13387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:31.390688 13396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88f031632c9045ffb3686ee382c62823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f031632c9045ffb3686ee382c62823" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43683 } } }
I20260811 03:01:31.390826 13396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:31.391119 13397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88f031632c9045ffb3686ee382c62823. Latest consensus state: current_term: 1 leader_uuid: "88f031632c9045ffb3686ee382c62823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88f031632c9045ffb3686ee382c62823" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43683 } } }
I20260811 03:01:31.391211 13397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:31.391379 13409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:31.394243 13409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:31.399363 13409 catalog_manager.cc:1347] Generated new cluster ID: 85fad7766bb843cf9b96bf54680a06e5
I20260811 03:01:31.399425 13409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:31.431344 13409 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:31.432602 13409 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:31.442188 13409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823: Generated new TSK 0
I20260811 03:01:31.442967 13409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:31.446511 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43683
--builtin_ntp_servers=127.8.203.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:01:31.555583 13427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:31.555811 13427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:31.555861 13427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:31.559772 13427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:31.559819 13427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:31.559898 13427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:01:31.564404 13427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:31.565457 13427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:31.566761 13427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:01:31.575865 13427 server_base.cc:1034] running on GCE node
W20260811 03:01:31.575824 13435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:31.575827 13438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:31.575829 13436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:31.576424 13427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:31.576999 13427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:31.578137 13427 hybrid_clock.cc:648] HybridClock initialized: now 1786417291578113 us; error 36 us; skew 500 ppm
I20260811 03:01:31.579778 13427 webserver.cc:460] Webserver started at http://127.8.203.65:33125/ using document root <none> and password file <none>
I20260811 03:01:31.580304 13427 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:31.580374 13427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:31.580575 13427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:31.582571 13427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "d7685dc59e4f49b1a528f616ae591d27"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "7ab94dde0bcee096b4419234e112fe42"
server_key_iv: "d5e4abf5b3d78d36cf4232e8c1ca45bf"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.583103 13427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "d7685dc59e4f49b1a528f616ae591d27"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "7ab94dde0bcee096b4419234e112fe42"
server_key_iv: "d5e4abf5b3d78d36cf4232e8c1ca45bf"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.586607 13427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:01:31.588836 13452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.590019 13427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:01:31.590123 13427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "d7685dc59e4f49b1a528f616ae591d27"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "7ab94dde0bcee096b4419234e112fe42"
server_key_iv: "d5e4abf5b3d78d36cf4232e8c1ca45bf"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.590220 13427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:31.606173 13427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:31.606758 13427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:31.606917 13427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:31.607451 13427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:31.608301 13427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:31.608355 13427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.608407 13427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:31.608435 13427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.616611 13427 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:33633
I20260811 03:01:31.616664 13679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:33633 every 8 connection(s)
I20260811 03:01:31.617471 13427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:01:31.623078 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 13427
I20260811 03:01:31.623171 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:01:31.623479 9005 external_mini_cluster.cc:1428] Setting key 509367f421e4cabc9e6bb81ecb38d468
I20260811 03:01:31.625514 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43683
--builtin_ntp_servers=127.8.203.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:01:31.626724 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.619137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:46223 (local address 127.8.203.126:43683)
0811 03:01:31.619268 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.619271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.620236 (+ 965us) server_negotiation.cc:407] Connection header received
0811 03:01:31.621622 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.621625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.621682 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.621753 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:31.622617 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.623161 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.624593 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.624938 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.625021 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.626266 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:31.626270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:31.626273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:31.626285 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:31.626309 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:31.626318 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:31.626397 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:31.626530 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 03:01:31.626586 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 03:01:31.627686 13686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.619497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43683 (local address 127.8.203.65:46223)
0811 03:01:31.620104 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:31.620143 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:31.621420 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:31.621930 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:31.621937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:31.622019 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:31.622475 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.622488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.623300 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.623304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.624482 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.624493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.625093 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.625098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.625314 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.625957 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:31.625959 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:31.626007 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:31.626109 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:31.626402 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:31.626409 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:31.626414 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 03:01:31.626526 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 03:01:31.626801 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":110,"threads_started":1}
I20260811 03:01:31.628881 13680 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43683
I20260811 03:01:31.629086 13680 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:31.629590 13680 heartbeater.cc:511] Master 127.8.203.126:43683 requested a full tablet report, sending...
I20260811 03:01:31.630975 13310 ts_manager.cc:194] Registered new tserver with Master: d7685dc59e4f49b1a528f616ae591d27 (127.8.203.65:33633)
I20260811 03:01:31.632071 13310 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.65:46223
W20260811 03:01:31.738906 13688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:31.739164 13688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:31.739205 13688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:31.743176 13688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:31.743227 13688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:31.743291 13688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:01:31.747931 13688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:31.749006 13688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:31.750413 13688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:31.759701 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:31.759766 13704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:31.759874 13688 server_base.cc:1034] running on GCE node
W20260811 03:01:31.760010 13706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:31.760394 13688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:31.761019 13688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:31.762161 13688 hybrid_clock.cc:648] HybridClock initialized: now 1786417291762127 us; error 46 us; skew 500 ppm
I20260811 03:01:31.763825 13688 webserver.cc:460] Webserver started at http://127.8.203.66:40853/ using document root <none> and password file <none>
I20260811 03:01:31.764356 13688 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:31.764415 13688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:31.764582 13688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:31.766834 13688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "e02f8b035fb04cb88afda34f82d47e11"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "56b0286eed97a21a8033d18c97f363de"
server_key_iv: "a269377d95ffe4acca29c7afaa66fe98"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.767331 13688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "e02f8b035fb04cb88afda34f82d47e11"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "56b0286eed97a21a8033d18c97f363de"
server_key_iv: "a269377d95ffe4acca29c7afaa66fe98"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.770764 13688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:01:31.773077 13718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.774210 13688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:31.774312 13688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "e02f8b035fb04cb88afda34f82d47e11"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "56b0286eed97a21a8033d18c97f363de"
server_key_iv: "a269377d95ffe4acca29c7afaa66fe98"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.774433 13688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:31.791426 13688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:31.791992 13688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:31.792146 13688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:31.792697 13688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:31.793639 13688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:31.793695 13688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.793747 13688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:31.793772 13688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.802440 13688 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:38363
I20260811 03:01:31.802464 13931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:38363 every 8 connection(s)
I20260811 03:01:31.803293 13688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:01:31.811570 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.804948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:34393 (local address 127.8.203.126:43683)
0811 03:01:31.805072 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.805075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.806140 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 03:01:31.807476 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.807478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.807524 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.807591 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:31.808427 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.808946 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.810005 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.810217 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.810285 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.811236 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:31.811238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:31.811240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:31.811250 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:31.811264 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:31.811270 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:31.811312 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:31.811416 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 03:01:31.811460 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:01:31.812362 13936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.805359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43683 (local address 127.8.203.66:34393)
0811 03:01:31.806015 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:31.806054 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:31.807302 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:31.807754 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:31.807760 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:31.807841 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:31.808321 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.808334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.809034 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.809036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.809908 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.809917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.810323 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.810325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.810457 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.811019 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:31.811021 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:31.811060 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:31.811120 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:31.811321 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:31.811326 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:31.811330 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 03:01:31.811412 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 03:01:31.811613 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":112,"threads_started":1}
I20260811 03:01:31.813155 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 13688
I20260811 03:01:31.813261 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:01:31.813496 13932 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43683
I20260811 03:01:31.813601 9005 external_mini_cluster.cc:1428] Setting key 7c9a0244c7bd8830aa19fba6bdd949f4
I20260811 03:01:31.813714 13932 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:31.814208 13932 heartbeater.cc:511] Master 127.8.203.126:43683 requested a full tablet report, sending...
I20260811 03:01:31.815171 13310 ts_manager.cc:194] Registered new tserver with Master: e02f8b035fb04cb88afda34f82d47e11 (127.8.203.66:38363)
I20260811 03:01:31.815507 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43683
--builtin_ntp_servers=127.8.203.84:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:01:31.815855 13310 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.66:34393
W20260811 03:01:31.926651 13937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:31.926898 13937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:31.926936 13937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:31.930843 13937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:31.930888 13937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:31.930953 13937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:01:31.935528 13937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:43683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:31.936532 13937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:31.937917 13937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:31.947117 13957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:31.947126 13958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:31.947227 13937 server_base.cc:1034] running on GCE node
W20260811 03:01:31.947126 13960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:31.947706 13937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:31.948261 13937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:31.949401 13937 hybrid_clock.cc:648] HybridClock initialized: now 1786417291949383 us; error 27 us; skew 500 ppm
I20260811 03:01:31.951061 13937 webserver.cc:460] Webserver started at http://127.8.203.67:35297/ using document root <none> and password file <none>
I20260811 03:01:31.951606 13937 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:31.951665 13937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:31.951869 13937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:31.953864 13937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "622fdbd8faa14827947d5e1fea0345b8"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "58435667498ae2c2775ddd4e792b5b53"
server_key_iv: "4db33f41fe7daf8b0af1345ddfb926bf"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.954396 13937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "622fdbd8faa14827947d5e1fea0345b8"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "58435667498ae2c2775ddd4e792b5b53"
server_key_iv: "4db33f41fe7daf8b0af1345ddfb926bf"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.957957 13937 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:01:31.960222 13969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.961372 13937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:01:31.961472 13937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "622fdbd8faa14827947d5e1fea0345b8"
format_stamp: "Formatted at 2026-08-11 03:01:31 on dist-test-slave-gjw7"
server_key: "58435667498ae2c2775ddd4e792b5b53"
server_key_iv: "4db33f41fe7daf8b0af1345ddfb926bf"
server_key_version: "encryptionkey@0"
I20260811 03:01:31.961570 13937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:31.970219 13937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:31.970784 13937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:31.970932 13937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:31.971444 13937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:31.972311 13937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:31.972363 13937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.972415 13937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:31.972445 13937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:31.980731 13937 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:33297
I20260811 03:01:31.980754 14174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:33297 every 8 connection(s)
I20260811 03:01:31.981586 13937 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:01:31.981843 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 13937
I20260811 03:01:31.981962 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:01:31.982267 9005 external_mini_cluster.cc:1428] Setting key 72697c4d63a0c8e85d77f76453017179
I20260811 03:01:31.990651 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.983349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:47957 (local address 127.8.203.126:43683)
0811 03:01:31.983482 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.983486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.984722 (+ 1236us) server_negotiation.cc:407] Connection header received
0811 03:01:31.986174 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.986177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.986232 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.986301 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:31.987183 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.987720 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.988819 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.989045 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.989131 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.990164 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:31.990167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:31.990170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:31.990184 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:31.990202 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:31.990209 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:31.990265 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:31.990405 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 03:01:31.990487 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:01:31.991457 14179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.983834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43683 (local address 127.8.203.67:47957)
0811 03:01:31.984598 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:31.984638 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:31.985946 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:31.986482 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:31.986489 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:31.986574 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:31.987051 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.987065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.987843 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.987846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.988703 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:31.988712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.989178 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.989182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:31.989381 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:31.989923 (+ 542us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:31.989925 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:31.989963 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:31.990026 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:31.990275 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:31.990284 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:31.990290 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 03:01:31.990399 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 03:01:31.990665 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":138,"threads_started":1}
I20260811 03:01:31.992429 14176 heartbeater.cc:347] Connected to a master server at 127.8.203.126:43683
I20260811 03:01:31.992641 14176 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:31.993124 14176 heartbeater.cc:511] Master 127.8.203.126:43683 requested a full tablet report, sending...
I20260811 03:01:31.994124 13310 ts_manager.cc:194] Registered new tserver with Master: 622fdbd8faa14827947d5e1fea0345b8 (127.8.203.67:33297)
I20260811 03:01:31.994685 13310 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.67:47957
I20260811 03:01:31.996202 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:01:32.001247 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:31.998077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.8.203.126:43683)
0811 03:01:31.998194 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:31.998197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:31.998332 (+ 135us) server_negotiation.cc:407] Connection header received
0811 03:01:31.998426 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:31.998428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:31.998462 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:31.998523 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:31.998718 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:31.999238 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:31.999975 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.000177 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.000261 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.000890 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:32.000893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:32.000895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:32.000906 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:32.000923 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:32.000930 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:32.001029 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:32.001097 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 03:01:32.001129 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:01:32.003682 13310 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:01:32.006528 13310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:01:32.020458 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.015342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38363 (local address 127.0.0.1:43476)
0811 03:01:32.015578 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:32.015595 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:32.015675 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:32.016587 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:32.016590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:32.016608 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:32.016712 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.016719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.017701 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.017704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.018944 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.018955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.019942 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.019946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.020052 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.020095 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 03:01:32.020198 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 03:01:32.020315 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260811 03:01:32.020480 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.015496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.8.203.66:38363)
0811 03:01:32.016183 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:32.016193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:32.016214 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:01:32.016285 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:32.016290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:32.016471 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:32.016602 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:32.016816 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.017600 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.019061 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.019821 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.019942 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.020138 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:32.020225 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 03:01:32.020320 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":125,"threads_started":1}
I20260811 03:01:32.020610 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.014779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:33297 (local address 127.0.0.1:40534)
0811 03:01:32.015145 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:32.015159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:32.015781 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:32.016414 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:32.016420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:32.016441 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:32.016565 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.016571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.018044 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.018048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.019115 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.019123 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.020223 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.020226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.020315 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.020338 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 03:01:32.020420 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 03:01:32.020489 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"spinlock_wait_cycles":4352,"thread_start_us":98,"threads_started":1}
I20260811 03:01:32.020956 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.015630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.8.203.67:33297)
0811 03:01:32.015911 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:32.015940 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:32.015964 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:01:32.016036 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:32.016043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:32.016246 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:32.016397 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:32.016695 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.017928 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.019213 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.019942 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.020027 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.020162 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:32.020742 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 03:01:32.020818 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
I20260811 03:01:32.021338 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.014629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:33633 (local address 127.0.0.1:37878)
0811 03:01:32.015104 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:32.015158 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:32.015294 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:32.016553 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:32.016555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:32.016570 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:32.016644 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.016649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.019205 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.019207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.019888 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.019896 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.020932 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.020936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.021025 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.021051 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 03:01:32.021114 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 03:01:32.021183 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"spinlock_wait_cycles":4352,"thread_start_us":79,"threads_started":1}
I20260811 03:01:32.021407 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.014760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37878 (local address 127.8.203.65:33633)
0811 03:01:32.016049 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:32.016060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:32.016084 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:01:32.016146 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:32.016150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:32.016286 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:32.016414 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:32.016760 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.017928 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.020020 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.020806 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.020915 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.021082 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:32.021134 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 03:01:32.021213 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":101,"threads_started":1}
I20260811 03:01:32.023810 13799 tablet_service.cc:1467] Processing CreateTablet for tablet f15622968f9d4419870fd52603878883 (DEFAULT_TABLE table=test-table [id=b3fd53f59ace4cbc8ebb57bcf9223d3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:32.024032 14054 tablet_service.cc:1467] Processing CreateTablet for tablet f15622968f9d4419870fd52603878883 (DEFAULT_TABLE table=test-table [id=b3fd53f59ace4cbc8ebb57bcf9223d3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:32.024032 13547 tablet_service.cc:1467] Processing CreateTablet for tablet f15622968f9d4419870fd52603878883 (DEFAULT_TABLE table=test-table [id=b3fd53f59ace4cbc8ebb57bcf9223d3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:32.024824 13799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15622968f9d4419870fd52603878883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:32.025040 14054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15622968f9d4419870fd52603878883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:32.025040 13547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15622968f9d4419870fd52603878883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:32.031430 14210 tablet_bootstrap.cc:492] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Bootstrap starting.
I20260811 03:01:32.032462 14211 tablet_bootstrap.cc:492] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Bootstrap starting.
I20260811 03:01:32.033587 14213 tablet_bootstrap.cc:492] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27: Bootstrap starting.
I20260811 03:01:32.033826 14210 tablet_bootstrap.cc:654] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:32.034610 14211 tablet_bootstrap.cc:654] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:32.034696 14210 log.cc:824] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:32.035328 14213 tablet_bootstrap.cc:654] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:32.035372 14211 log.cc:824] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:32.036095 14213 log.cc:824] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:32.036906 14210 tablet_bootstrap.cc:492] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: No bootstrap required, opened a new log
I20260811 03:01:32.037096 14210 ts_tablet_manager.cc:1397] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:01:32.037695 14211 tablet_bootstrap.cc:492] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: No bootstrap required, opened a new log
I20260811 03:01:32.037792 14213 tablet_bootstrap.cc:492] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27: No bootstrap required, opened a new log
I20260811 03:01:32.037892 14211 ts_tablet_manager.cc:1397] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 03:01:32.037981 14213 ts_tablet_manager.cc:1397] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:01:32.040890 14213 raft_consensus.cc:348] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.041096 14213 raft_consensus.cc:374] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:32.041147 14213 raft_consensus.cc:729] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7685dc59e4f49b1a528f616ae591d27, State: Initialized, Role: FOLLOWER
I20260811 03:01:32.041353 14210 raft_consensus.cc:348] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.041606 14210 raft_consensus.cc:374] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:32.041664 14210 raft_consensus.cc:729] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02f8b035fb04cb88afda34f82d47e11, State: Initialized, Role: FOLLOWER
I20260811 03:01:32.041601 14213 consensus_queue.cc:260] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.041903 14211 raft_consensus.cc:348] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.042133 14211 raft_consensus.cc:374] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:32.042074 14210 consensus_queue.cc:260] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.042182 14211 raft_consensus.cc:729] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622fdbd8faa14827947d5e1fea0345b8, State: Initialized, Role: FOLLOWER
I20260811 03:01:32.042367 13680 heartbeater.cc:503] Master 127.8.203.126:43683 was elected leader, sending a full tablet report...
I20260811 03:01:32.042578 14213 ts_tablet_manager.cc:1428] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:32.042621 14211 consensus_queue.cc:260] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.043018 14210 ts_tablet_manager.cc:1428] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 03:01:32.043114 13932 heartbeater.cc:503] Master 127.8.203.126:43683 was elected leader, sending a full tablet report...
I20260811 03:01:32.043946 14176 heartbeater.cc:503] Master 127.8.203.126:43683 was elected leader, sending a full tablet report...
I20260811 03:01:32.044019 14211 ts_tablet_manager.cc:1428] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 03:01:32.054103 13935 tablet.cc:2369] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:01:32.118572 13684 tablet.cc:2369] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:01:32.232749 14178 tablet.cc:2369] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:01:32.249254 14219 raft_consensus.cc:482] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:32.249513 14219 raft_consensus.cc:504] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.250605 14219 leader_election.cc:288] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 622fdbd8faa14827947d5e1fea0345b8 (127.8.203.67:33297), e02f8b035fb04cb88afda34f82d47e11 (127.8.203.66:38363)
I20260811 03:01:32.255235 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.250908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:46477 (local address 127.8.203.67:33297)
0811 03:01:32.251077 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:32.251082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:32.251152 (+ 70us) server_negotiation.cc:407] Connection header received
0811 03:01:32.251305 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:32.251308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:32.251361 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:32.251428 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:32.251711 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.252467 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.253957 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.254593 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.254712 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.254762 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:32.255036 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 03:01:32.255106 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:01:32.255235 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.251279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:36899 (local address 127.8.203.66:38363)
0811 03:01:32.251418 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:32.251422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:32.251558 (+ 136us) server_negotiation.cc:407] Connection header received
0811 03:01:32.251661 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:32.251664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:32.251715 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:32.251785 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:32.252007 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.252689 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.254008 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.254592 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.254712 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.254762 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:32.255036 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 03:01:32.255106 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 03:01:32.255292 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.250807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:33297 (local address 127.8.203.65:46477)
0811 03:01:32.251068 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:32.251082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:32.251191 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:32.251460 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:32.251463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:32.251489 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:32.251612 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.251618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.252569 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.252572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.253838 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.253851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.254711 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.254715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.254829 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.254936 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 03:01:32.255011 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 03:01:32.255098 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":75,"thread_start_us":82,"threads_started":1}
I20260811 03:01:32.255296 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.251172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:38363 (local address 127.8.203.65:36899)
0811 03:01:32.251477 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:32.251489 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:32.251576 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:32.251799 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:32.251801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:32.251817 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:32.251918 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.251922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.252774 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.252777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.253908 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:32.253917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.254701 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.254704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:32.254881 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.254941 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 03:01:32.255011 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 03:01:32.255098 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":51,"thread_start_us":77,"threads_started":1}
I20260811 03:01:32.255970 13827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15622968f9d4419870fd52603878883" candidate_uuid: "d7685dc59e4f49b1a528f616ae591d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02f8b035fb04cb88afda34f82d47e11" is_pre_election: true
I20260811 03:01:32.256058 14100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15622968f9d4419870fd52603878883" candidate_uuid: "d7685dc59e4f49b1a528f616ae591d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622fdbd8faa14827947d5e1fea0345b8" is_pre_election: true
I20260811 03:01:32.256256 13827 raft_consensus.cc:2454] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7685dc59e4f49b1a528f616ae591d27 in term 0.
I20260811 03:01:32.256314 14100 raft_consensus.cc:2454] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7685dc59e4f49b1a528f616ae591d27 in term 0.
I20260811 03:01:32.256719 13460 leader_election.cc:302] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d7685dc59e4f49b1a528f616ae591d27, e02f8b035fb04cb88afda34f82d47e11; no voters:
I20260811 03:01:32.256973 14219 raft_consensus.cc:2781] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:01:32.257053 14219 raft_consensus.cc:482] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:01:32.257083 14219 raft_consensus.cc:3037] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:32.258118 14219 raft_consensus.cc:504] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.258435 14219 leader_election.cc:288] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [CANDIDATE]: Term 1 election: Requested vote from peers 622fdbd8faa14827947d5e1fea0345b8 (127.8.203.67:33297), e02f8b035fb04cb88afda34f82d47e11 (127.8.203.66:38363)
I20260811 03:01:32.258728 14100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15622968f9d4419870fd52603878883" candidate_uuid: "d7685dc59e4f49b1a528f616ae591d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622fdbd8faa14827947d5e1fea0345b8"
I20260811 03:01:32.258806 13827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15622968f9d4419870fd52603878883" candidate_uuid: "d7685dc59e4f49b1a528f616ae591d27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02f8b035fb04cb88afda34f82d47e11"
I20260811 03:01:32.258826 14100 raft_consensus.cc:3037] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:32.258932 13827 raft_consensus.cc:3037] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:32.259784 14100 raft_consensus.cc:2454] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7685dc59e4f49b1a528f616ae591d27 in term 1.
I20260811 03:01:32.259941 13827 raft_consensus.cc:2454] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7685dc59e4f49b1a528f616ae591d27 in term 1.
I20260811 03:01:32.260061 13460 leader_election.cc:302] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 622fdbd8faa14827947d5e1fea0345b8, d7685dc59e4f49b1a528f616ae591d27; no voters:
I20260811 03:01:32.260207 14219 raft_consensus.cc:2781] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:32.260421 14219 raft_consensus.cc:686] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [term 1 LEADER]: Becoming Leader. State: Replica: d7685dc59e4f49b1a528f616ae591d27, State: Running, Role: LEADER
I20260811 03:01:32.260735 14219 consensus_queue.cc:237] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } }
I20260811 03:01:32.263731 13310 catalog_manager.cc:5495] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7685dc59e4f49b1a528f616ae591d27 (127.8.203.65). New cstate: current_term: 1 leader_uuid: "d7685dc59e4f49b1a528f616ae591d27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7685dc59e4f49b1a528f616ae591d27" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 } health_report { overall_health: UNKNOWN } } }
I20260811 03:01:32.310102 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.306931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37882 (local address 127.8.203.65:33633)
0811 03:01:32.307055 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:32.307059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:32.307073 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:01:32.307179 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:32.307181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:32.307240 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:32.307325 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:01:32.307507 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.307986 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.308784 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:32.309005 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:32.309075 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:32.309339 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:01:32.309748 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:01:32.309860 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:32.309924 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 03:01:32.309958 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:01:32.315733 13827 raft_consensus.cc:1261] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 1 FOLLOWER]: Refusing update from remote peer d7685dc59e4f49b1a528f616ae591d27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:32.315743 14100 raft_consensus.cc:1261] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 1 FOLLOWER]: Refusing update from remote peer d7685dc59e4f49b1a528f616ae591d27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:32.316394 14219 consensus_queue.cc:1035] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "622fdbd8faa14827947d5e1fea0345b8" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:32.316589 14228 consensus_queue.cc:1035] T f15622968f9d4419870fd52603878883 P d7685dc59e4f49b1a528f616ae591d27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02f8b035fb04cb88afda34f82d47e11" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:32.326246 14237 mvcc.cc:205] Tried to move back new op lower bound from 7317165229318594560 to 7317165229102460928. Current Snapshot: MvccSnapshot[applied={T|T < 7317165229318594560}]
I20260811 03:01:32.331525 13307 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:32.331660 13307 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:32.333735 13307 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88f031632c9045ffb3686ee382c62823: Sending DeleteTablet for 3 replicas of tablet f15622968f9d4419870fd52603878883
I20260811 03:01:32.334399 13547 tablet_service.cc:1514] Processing DeleteTablet for tablet f15622968f9d4419870fd52603878883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:32 UTC) from {username='slave'} at 127.0.0.1:37878
I20260811 03:01:32.334506 13799 tablet_service.cc:1514] Processing DeleteTablet for tablet f15622968f9d4419870fd52603878883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:32 UTC) from {username='slave'} at 127.0.0.1:43476
I20260811 03:01:32.334585 14054 tablet_service.cc:1514] Processing DeleteTablet for tablet f15622968f9d4419870fd52603878883 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:32 UTC) from {username='slave'} at 127.0.0.1:40534
I20260811 03:01:32.335026 14257 tablet_replica.cc:330] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: stopping tablet replica
I20260811 03:01:32.335196 14256 tablet_replica.cc:330] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: stopping tablet replica
I20260811 03:01:32.335354 14257 raft_consensus.cc:2229] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:32.335568 14256 raft_consensus.cc:2229] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:32.335633 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 13427
I20260811 03:01:32.335654 14257 raft_consensus.cc:2258] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:32.335974 14256 raft_consensus.cc:2258] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:32.340210 14256 ts_tablet_manager.cc:1905] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:32.340898 14257 ts_tablet_manager.cc:1905] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:32.342934 14256 ts_tablet_manager.cc:1918] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:32.343034 14256 log.cc:1196] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/f15622968f9d4419870fd52603878883
I20260811 03:01:32.343019 14257 ts_tablet_manager.cc:1918] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:32.343104 14257 log.cc:1196] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/f15622968f9d4419870fd52603878883
I20260811 03:01:32.343325 14256 ts_tablet_manager.cc:1939] T f15622968f9d4419870fd52603878883 P e02f8b035fb04cb88afda34f82d47e11: Deleting consensus metadata
I20260811 03:01:32.343379 14257 ts_tablet_manager.cc:1939] T f15622968f9d4419870fd52603878883 P 622fdbd8faa14827947d5e1fea0345b8: Deleting consensus metadata
I20260811 03:01:32.344491 13284 catalog_manager.cc:4841] TS e02f8b035fb04cb88afda34f82d47e11 (127.8.203.66:38363): tablet f15622968f9d4419870fd52603878883 (table test-table [id=b3fd53f59ace4cbc8ebb57bcf9223d3a]) successfully deleted
I20260811 03:01:32.344671 13284 catalog_manager.cc:4841] TS 622fdbd8faa14827947d5e1fea0345b8 (127.8.203.67:33297): tablet f15622968f9d4419870fd52603878883 (table test-table [id=b3fd53f59ace4cbc8ebb57bcf9223d3a]) successfully deleted
W20260811 03:01:32.347527 13280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:33633 (error 108)
I20260811 03:01:32.348255 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 13688
I20260811 03:01:32.348489 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:32.348196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:33633 (local address 127.0.0.1:37884)
0811 03:01:32.348329 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:32.348410 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:33633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 03:01:32.348634 13280 catalog_manager.cc:4584] TS d7685dc59e4f49b1a528f616ae591d27 (127.8.203.65:33633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f15622968f9d4419870fd52603878883: Network error: Client connection negotiation failed: client connection to 127.8.203.65:33633: connect: Connection refused (error 111)
I20260811 03:01:32.364054 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 13937
I20260811 03:01:32.387084 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 13253
2026-08-11T03:01:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1246 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:32 dist-test-slave-gjw7 krb5kdc[14262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:32 dist-test-slave-gjw7 krb5kdc[14262](info): set up 2 sockets
Aug 11 03:01:32 dist-test-slave-gjw7 krb5kdc[14262](info): commencing operation
krb5kdc: starting...
W20260811 03:01:33.577916 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:33 dist-test-slave-gjw7 krb5kdc[14262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:01:33.693634 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:42845
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:42845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:33.802700 14284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:33.803020 14284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:33.803079 14284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:33.807104 14284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:33.807152 14284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:33.807170 14284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:33.807185 14284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:33.807204 14284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:33.812213 14284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:42845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:33.813112 14284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:33.814035 14284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:33.822005 14291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:33.822036 14293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:33.822005 14296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:33.822299 14284 server_base.cc:1034] running on GCE node
I20260811 03:01:33.822808 14284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:33.823825 14284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:33.824986 14284 hybrid_clock.cc:648] HybridClock initialized: now 1786417293824971 us; error 25 us; skew 500 ppm
Aug 11 03:01:33 dist-test-slave-gjw7 krb5kdc[14262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417293, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:33.827623 14284 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:01:33.828469 14284 webserver.cc:460] Webserver started at http://127.8.203.126:37243/ using document root <none> and password file <none>
I20260811 03:01:33.828981 14284 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:33.829041 14284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:33.829236 14284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:33.831176 14284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "e72920f121ac4e57a1f78cda53d2ce98"
format_stamp: "Formatted at 2026-08-11 03:01:33 on dist-test-slave-gjw7"
server_key: "22bd5e08b3ed9d68b83e4dc79b0b79b3"
server_key_iv: "8d817a57f5b477619fe4aef9e8c24d9b"
server_key_version: "encryptionkey@0"
I20260811 03:01:33.831701 14284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "e72920f121ac4e57a1f78cda53d2ce98"
format_stamp: "Formatted at 2026-08-11 03:01:33 on dist-test-slave-gjw7"
server_key: "22bd5e08b3ed9d68b83e4dc79b0b79b3"
server_key_iv: "8d817a57f5b477619fe4aef9e8c24d9b"
server_key_version: "encryptionkey@0"
I20260811 03:01:33.835294 14284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:01:33.837554 14307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:33.838714 14284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 03:01:33.838824 14284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "e72920f121ac4e57a1f78cda53d2ce98"
format_stamp: "Formatted at 2026-08-11 03:01:33 on dist-test-slave-gjw7"
server_key: "22bd5e08b3ed9d68b83e4dc79b0b79b3"
server_key_iv: "8d817a57f5b477619fe4aef9e8c24d9b"
server_key_version: "encryptionkey@0"
I20260811 03:01:33.838923 14284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:33.850302 14284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:33.852666 14284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:33.852820 14284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:33.859319 14284 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:42845
I20260811 03:01:33.859333 14408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:42845 every 8 connection(s)
I20260811 03:01:33.860183 14284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:33.862757 14412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:33.868021 14412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98: Bootstrap starting.
I20260811 03:01:33.868781 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 14284
I20260811 03:01:33.868932 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:33.869238 9005 external_mini_cluster.cc:1428] Setting key 0897742299c7b742921467edb1215399
I20260811 03:01:33.870868 14412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:33.871567 14412 log.cc:824] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:33.873623 14412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98: No bootstrap required, opened a new log
Aug 11 03:01:33 dist-test-slave-gjw7 krb5kdc[14262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:33.877810 14412 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72920f121ac4e57a1f78cda53d2ce98" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42845 } }
I20260811 03:01:33.878089 14412 raft_consensus.cc:374] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:33.878149 14412 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72920f121ac4e57a1f78cda53d2ce98, State: Initialized, Role: FOLLOWER
I20260811 03:01:33.878590 14412 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72920f121ac4e57a1f78cda53d2ce98" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42845 } }
I20260811 03:01:33.878753 14412 raft_consensus.cc:388] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:33.878820 14412 raft_consensus.cc:482] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:33.878926 14412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:33.879971 14412 raft_consensus.cc:504] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72920f121ac4e57a1f78cda53d2ce98" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42845 } }
I20260811 03:01:33.880328 14412 leader_election.cc:302] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e72920f121ac4e57a1f78cda53d2ce98; no voters:
I20260811 03:01:33.880666 14412 leader_election.cc:288] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:33.880740 14420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:33.880954 14420 raft_consensus.cc:686] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [term 1 LEADER]: Becoming Leader. State: Replica: e72920f121ac4e57a1f78cda53d2ce98, State: Running, Role: LEADER
I20260811 03:01:33.881263 14420 consensus_queue.cc:237] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72920f121ac4e57a1f78cda53d2ce98" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42845 } }
I20260811 03:01:33.882011 14412 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:33.882642 14420 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e72920f121ac4e57a1f78cda53d2ce98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72920f121ac4e57a1f78cda53d2ce98" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42845 } } }
I20260811 03:01:33.882836 14420 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:33.882907 14421 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e72920f121ac4e57a1f78cda53d2ce98. Latest consensus state: current_term: 1 leader_uuid: "e72920f121ac4e57a1f78cda53d2ce98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e72920f121ac4e57a1f78cda53d2ce98" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 42845 } } }
I20260811 03:01:33.883001 14421 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:33.885306 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:33.870363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.8.203.126:42845)
0811 03:01:33.870733 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:33.870749 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:33.870784 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:01:33.871410 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:33.871428 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:33.871761 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:33.872085 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:33.872831 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:33.873849 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:33.874964 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:33.875409 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:33.875518 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:33.878300 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:33.878305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:33.878318 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:33.878344 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:33.880506 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:33.881506 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:33.881512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:33.881518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:33.881862 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:33.882423 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:33.882428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:33.882430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:33.883235 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:33.883417 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:33.883982 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 03:01:33.884299 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":102,"threads_started":1}
I20260811 03:01:33.886186 14426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:33.889326 14426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:33.894737 14426 catalog_manager.cc:1347] Generated new cluster ID: ed24f0d01c47476f8696bfd2d797ecaf
I20260811 03:01:33.894788 14426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:33.904735 14426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:33.905630 14426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:33.911856 14426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98: Generated new TSK 0
I20260811 03:01:33.912451 14426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.203.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
Entry for principal kudu/127.8.203.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab.
I20260811 03:01:33.938330 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42845
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:34.047562 14460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:34.047798 14460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:34.047838 14460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:34.051692 14460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:34.051738 14460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:34.051790 14460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:01:34.056687 14460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.65.keytab
--principal=kudu/127.8.203.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:34.057611 14460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:34.058525 14460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:34.067703 14478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:34.067698 14473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:34.067749 14460 server_base.cc:1034] running on GCE node
W20260811 03:01:34.067723 14475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:34.068332 14460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:34.068955 14460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:34.070107 14460 hybrid_clock.cc:648] HybridClock initialized: now 1786417294070093 us; error 25 us; skew 500 ppm
Aug 11 03:01:34 dist-test-slave-gjw7 krb5kdc[14262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:34.072608 14460 init.cc:377] Logged in from keytab as kudu/127.8.203.65@KRBTEST.COM (short username kudu)
I20260811 03:01:34.073485 14460 webserver.cc:460] Webserver started at http://127.8.203.65:40783/ using document root <none> and password file <none>
I20260811 03:01:34.074028 14460 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:34.074091 14460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:34.074277 14460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:34.076275 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "0df1e73442f2490a971a7ee4bf303e36"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "402fa927465bbdd854fccfb70b2fdd91"
server_key_iv: "013731c0b18dcfbf7d401065c1c82c9a"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.076790 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "0df1e73442f2490a971a7ee4bf303e36"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "402fa927465bbdd854fccfb70b2fdd91"
server_key_iv: "013731c0b18dcfbf7d401065c1c82c9a"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.080309 14460 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 03:01:34.082578 14486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.083686 14460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:34.083799 14460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "0df1e73442f2490a971a7ee4bf303e36"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "402fa927465bbdd854fccfb70b2fdd91"
server_key_iv: "013731c0b18dcfbf7d401065c1c82c9a"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.083913 14460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:34.092953 14460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:34.095486 14460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:34.095651 14460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:34.096194 14460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:34.097065 14460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:34.097110 14460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.097162 14460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:34.097187 14460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.105454 14460 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:40239
I20260811 03:01:34.105489 14734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:40239 every 8 connection(s)
I20260811 03:01:34.106287 14460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:01:34.114151 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 14460
I20260811 03:01:34.114274 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:01:34.114594 9005 external_mini_cluster.cc:1428] Setting key 6a05830d6c7197f27ed6e59d2105f7bb
Aug 11 03:01:34 dist-test-slave-gjw7 krb5kdc[14262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.65@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:34.118921 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.107939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:57005 (local address 127.8.203.126:42845)
0811 03:01:34.108077 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.108081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.108891 (+ 810us) server_negotiation.cc:407] Connection header received
0811 03:01:34.110148 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.110151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.110198 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.110268 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:34.111321 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.111842 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.112862 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.113090 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.113159 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.115543 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:34.115547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:34.115549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:34.115574 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:34.116929 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.117443 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.117446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.117447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.117525 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.117844 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.117847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.117848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.118021 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:34.118103 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.118589 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.118728 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 03:01:34.119760 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.108320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42845 (local address 127.8.203.65:57005)
0811 03:01:34.108745 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.108786 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.109960 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.110419 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.110425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.110751 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:34.111210 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.111224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.111928 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.111930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.112775 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.112783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.113195 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.113197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.113377 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.113843 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:34.113862 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:34.115403 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:34.117040 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:34.117046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:34.117060 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:34.117344 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:34.117608 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:34.117611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:34.117613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:34.117747 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:34.118118 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:34.118125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:34.118355 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.118510 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.118816 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":116,"threads_started":1}
I20260811 03:01:34.120962 14735 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42845
I20260811 03:01:34.121214 14735 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:34.121794 14735 heartbeater.cc:511] Master 127.8.203.126:42845 requested a full tablet report, sending...
I20260811 03:01:34.123529 14340 ts_manager.cc:194] Registered new tserver with Master: 0df1e73442f2490a971a7ee4bf303e36 (127.8.203.65:40239)
I20260811 03:01:34.125115 14340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.65@KRBTEST.COM'} at 127.8.203.65:57005
WARNING: no policy specified for kudu/127.8.203.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
Entry for principal kudu/127.8.203.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab.
I20260811 03:01:34.142356 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42845
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:34.250643 14740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:34.250874 14740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:34.250914 14740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:34.254796 14740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:34.254848 14740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:34.254902 14740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:01:34.259918 14740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.66.keytab
--principal=kudu/127.8.203.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:34.260857 14740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:34.261857 14740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:01:34.271160 14740 server_base.cc:1034] running on GCE node
W20260811 03:01:34.271133 14751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:34.271126 14748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:34.271127 14756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:34.271776 14740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:34.272364 14740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:34.273507 14740 hybrid_clock.cc:648] HybridClock initialized: now 1786417294273493 us; error 24 us; skew 500 ppm
Aug 11 03:01:34 dist-test-slave-gjw7 krb5kdc[14262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:34.276218 14740 init.cc:377] Logged in from keytab as kudu/127.8.203.66@KRBTEST.COM (short username kudu)
I20260811 03:01:34.276988 14740 webserver.cc:460] Webserver started at http://127.8.203.66:34377/ using document root <none> and password file <none>
I20260811 03:01:34.277556 14740 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:34.277614 14740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:34.277781 14740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:34.279680 14740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "364383b42cf04a5addddb7f850342b7c"
server_key_iv: "09501f5e1672b42bdd6b42123785218e"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.280165 14740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "364383b42cf04a5addddb7f850342b7c"
server_key_iv: "09501f5e1672b42bdd6b42123785218e"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.283710 14740 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:01:34.286051 14770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.287170 14740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:01:34.287282 14740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "364383b42cf04a5addddb7f850342b7c"
server_key_iv: "09501f5e1672b42bdd6b42123785218e"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.287369 14740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:34.320782 14740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:34.323437 14740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:34.323601 14740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:34.324177 14740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:34.324988 14740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:34.325035 14740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.325089 14740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:34.325115 14740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.333240 14740 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:33151
I20260811 03:01:34.333271 15015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:33151 every 8 connection(s)
I20260811 03:01:34.334112 14740 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:01:34.339376 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 14740
I20260811 03:01:34.339493 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:01:34.339759 9005 external_mini_cluster.cc:1428] Setting key 1c69a99e06da6070f7f79dd27a1e0156
Aug 11 03:01:34 dist-test-slave-gjw7 krb5kdc[14262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.66@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:34.347250 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.335713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:37053 (local address 127.8.203.126:42845)
0811 03:01:34.335826 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.335829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.336617 (+ 788us) server_negotiation.cc:407] Connection header received
0811 03:01:34.337955 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.337958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.338017 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.338088 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:34.339202 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.339785 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.340920 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.341138 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.341224 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.343847 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:34.343850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:34.343852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:34.343884 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:34.345226 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.345722 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.345725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.345727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.345820 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.346112 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.346115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.346117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.346356 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:34.346457 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.346964 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.347086 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 03:01:34.347946 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.336053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42845 (local address 127.8.203.66:37053)
0811 03:01:34.336477 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.336517 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.337747 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.338262 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.338269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.338610 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:34.339060 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.339073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.339936 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.339939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.340789 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.340798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.341272 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.341276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.341458 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.341979 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:34.342000 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:34.343693 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:34.345367 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:34.345372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:34.345383 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:34.345626 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:34.345899 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:34.345901 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:34.345902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:34.346022 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:34.346522 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:34.346527 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:34.346741 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.346888 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.347121 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":116,"threads_started":1}
I20260811 03:01:34.349002 15017 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42845
I20260811 03:01:34.349243 15017 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:34.349831 15017 heartbeater.cc:511] Master 127.8.203.126:42845 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.203.67@KRBTEST.COM; defaulting to no policy
I20260811 03:01:34.350935 14340 ts_manager.cc:194] Registered new tserver with Master: 3678c5c9bc254255b5ee2ffbd44dc0b7 (127.8.203.66:33151)
I20260811 03:01:34.351519 14340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.66@KRBTEST.COM'} at 127.8.203.66:37053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
Entry for principal kudu/127.8.203.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab.
I20260811 03:01:34.366454 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42845
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:34.475469 15028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:34.475682 15028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:34.475719 15028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:34.479559 15028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:34.479607 15028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:34.479658 15028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:01:34.484647 15028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.67.keytab
--principal=kudu/127.8.203.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:42845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:34.485550 15028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:34.486443 15028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:34.495534 15040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:34.495541 15044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:34.495575 15047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:34.495709 15028 server_base.cc:1034] running on GCE node
I20260811 03:01:34.496184 15028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:34.496745 15028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:34.497895 15028 hybrid_clock.cc:648] HybridClock initialized: now 1786417294497878 us; error 29 us; skew 500 ppm
Aug 11 03:01:34 dist-test-slave-gjw7 krb5kdc[14262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:34.500447 15028 init.cc:377] Logged in from keytab as kudu/127.8.203.67@KRBTEST.COM (short username kudu)
I20260811 03:01:34.501209 15028 webserver.cc:460] Webserver started at http://127.8.203.67:39049/ using document root <none> and password file <none>
I20260811 03:01:34.501736 15028 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:34.501796 15028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:34.501968 15028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:34.503896 15028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "9d0081ea27154dcaac0b971d16e1c0d2"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "f2b97b9b5d01573676e3a33e4dc37842"
server_key_iv: "11d3cede435396e3be8a1c537b2cc1a1"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.504392 15028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "9d0081ea27154dcaac0b971d16e1c0d2"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "f2b97b9b5d01573676e3a33e4dc37842"
server_key_iv: "11d3cede435396e3be8a1c537b2cc1a1"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.507893 15028 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 03:01:34.510164 15058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.511279 15028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:01:34.511375 15028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "9d0081ea27154dcaac0b971d16e1c0d2"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "f2b97b9b5d01573676e3a33e4dc37842"
server_key_iv: "11d3cede435396e3be8a1c537b2cc1a1"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.511456 15028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:34.529982 15028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:34.532894 15028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:34.533034 15028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:34.533553 15028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:34.534309 15028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:34.534359 15028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.534396 15028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:34.534416 15028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.542553 15028 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:33721
I20260811 03:01:34.542620 15285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:33721 every 8 connection(s)
I20260811 03:01:34.543406 15028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
Aug 11 03:01:34 dist-test-slave-gjw7 krb5kdc[14262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.67@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:34.552285 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15028
I20260811 03:01:34.552398 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:01:34.552693 9005 external_mini_cluster.cc:1428] Setting key d89351b1772b7d1c5cc9891467e95268
I20260811 03:01:34.556131 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.545076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:59635 (local address 127.8.203.126:42845)
0811 03:01:34.545212 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.545216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.546074 (+ 858us) server_negotiation.cc:407] Connection header received
0811 03:01:34.547314 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.547317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.547363 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.547444 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:34.548521 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.549038 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.550100 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.550304 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.550371 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.552623 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:34.552627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:34.552630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:34.552660 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:34.554166 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.554652 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.554655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.554656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.554736 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.555048 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.555051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.555053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.555242 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:34.555327 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.555829 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.555972 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 03:01:34.556905 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.545494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:42845 (local address 127.8.203.67:59635)
0811 03:01:34.545932 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.545973 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.547128 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.547585 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.547592 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.547948 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:34.548412 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.548426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.549122 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.549124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.550015 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.550024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.550405 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.550407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.550539 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.551068 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:01:34.551085 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:34.552447 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:34.554276 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:34.554282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:34.554298 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:34.554557 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:34.554819 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:01:34.554822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:01:34.554824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:01:34.554960 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:01:34.555351 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:34.555358 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:34.555600 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.555767 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.556102 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":116,"threads_started":1}
I20260811 03:01:34.557929 15286 heartbeater.cc:347] Connected to a master server at 127.8.203.126:42845
I20260811 03:01:34.558153 15286 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:34.558617 15286 heartbeater.cc:511] Master 127.8.203.126:42845 requested a full tablet report, sending...
I20260811 03:01:34.559597 14340 ts_manager.cc:194] Registered new tserver with Master: 9d0081ea27154dcaac0b971d16e1c0d2 (127.8.203.67:33721)
I20260811 03:01:34.560117 14340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.203.67@KRBTEST.COM'} at 127.8.203.67:59635
I20260811 03:01:34.566306 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:01:34.574860 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.568139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.8.203.126:42845)
0811 03:01:34.568263 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.568268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.568443 (+ 175us) server_negotiation.cc:407] Connection header received
0811 03:01:34.568562 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.568565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.568609 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.568672 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:34.569367 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.569943 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.570748 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.570973 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.571057 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.572260 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:34.572264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:34.572267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:34.572293 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:34.573345 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.573719 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.573722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.573724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.573807 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:34.574041 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:34.574045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:34.574046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:34.574260 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:34.574388 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.574603 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.574714 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 03:01:34.577461 14340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:01:34.580314 14340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:01:34.594362 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.588398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40239 (local address 127.0.0.1:48492)
0811 03:01:34.588713 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.588759 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.588878 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.590159 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.590162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.590191 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:34.590293 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.590300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.591904 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.591907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.593006 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.593015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.593925 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.593928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.593999 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.594017 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.594069 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.594209 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":17,"spinlock_wait_cycles":1792,"thread_start_us":72,"threads_started":1}
I20260811 03:01:34.594379 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.589174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.8.203.67:33721)
0811 03:01:34.589655 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.589666 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.589690 (+ 24us) server_negotiation.cc:407] Connection header received
0811 03:01:34.589768 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.589774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.589977 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.590097 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:34.590418 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.591251 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.592832 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.593676 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.593785 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.593940 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.594139 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.594238 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":172,"threads_started":1}
I20260811 03:01:34.594597 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.588513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48492 (local address 127.8.203.65:40239)
0811 03:01:34.589788 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.589801 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.589819 (+ 18us) server_negotiation.cc:407] Connection header received
0811 03:01:34.589882 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.589886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.590024 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.590168 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:34.590442 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.591545 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.593161 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.593792 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.594182 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.594318 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.594381 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.594464 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":116,"threads_started":1}
I20260811 03:01:34.594789 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.589094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:33151 (local address 127.0.0.1:33134)
0811 03:01:34.589675 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.589688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.589771 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.590807 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.590810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.590829 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:34.590913 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.590919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.592333 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.592336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.593211 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.593222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.594396 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.594400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.594507 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.594534 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.594598 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.594658 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260811 03:01:34.595009 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.588680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:33721 (local address 127.0.0.1:58902)
0811 03:01:34.589515 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.589529 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.589608 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.590131 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.590137 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.590160 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:34.590300 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.590308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.591360 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.591364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.592694 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.592706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.593807 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.593811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.593942 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.593988 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.594094 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.594727 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260811 03:01:34.595207 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.589257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.8.203.66:33151)
0811 03:01:34.590394 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.590405 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.590426 (+ 21us) server_negotiation.cc:407] Connection header received
0811 03:01:34.590496 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.590501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.590670 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.590812 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:34.591007 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.592193 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.593349 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.594087 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.594186 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.594317 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.594971 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.595066 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":148,"threads_started":1}
I20260811 03:01:34.597664 15149 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf78d56a22f40d5ad40675d8da2e623 (DEFAULT_TABLE table=test-table [id=088ac8b667b74768904084560df16097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:34.597664 14590 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf78d56a22f40d5ad40675d8da2e623 (DEFAULT_TABLE table=test-table [id=088ac8b667b74768904084560df16097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:34.598810 14590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf78d56a22f40d5ad40675d8da2e623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:34.598812 15149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf78d56a22f40d5ad40675d8da2e623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:34.602870 14886 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf78d56a22f40d5ad40675d8da2e623 (DEFAULT_TABLE table=test-table [id=088ac8b667b74768904084560df16097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:34.603716 14886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf78d56a22f40d5ad40675d8da2e623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:34.606081 15319 tablet_bootstrap.cc:492] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Bootstrap starting.
I20260811 03:01:34.606490 15322 tablet_bootstrap.cc:492] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36: Bootstrap starting.
I20260811 03:01:34.608848 15322 tablet_bootstrap.cc:654] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:34.610196 15322 log.cc:824] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:34.610872 15319 tablet_bootstrap.cc:654] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:34.610882 15323 tablet_bootstrap.cc:492] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Bootstrap starting.
I20260811 03:01:34.611815 15319 log.cc:824] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:34.611974 15322 tablet_bootstrap.cc:492] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36: No bootstrap required, opened a new log
I20260811 03:01:34.612160 15322 ts_tablet_manager.cc:1397] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 03:01:34.613427 15319 tablet_bootstrap.cc:492] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: No bootstrap required, opened a new log
I20260811 03:01:34.613600 15319 ts_tablet_manager.cc:1397] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 03:01:34.614291 15323 tablet_bootstrap.cc:654] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:34.615139 15323 log.cc:824] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:34.615901 15322 raft_consensus.cc:348] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.616173 15322 raft_consensus.cc:374] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:34.616230 15322 raft_consensus.cc:729] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df1e73442f2490a971a7ee4bf303e36, State: Initialized, Role: FOLLOWER
I20260811 03:01:34.616652 15322 consensus_queue.cc:260] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.616887 15319 raft_consensus.cc:348] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.617153 15319 raft_consensus.cc:374] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:34.617190 15323 tablet_bootstrap.cc:492] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: No bootstrap required, opened a new log
I20260811 03:01:34.617213 15319 raft_consensus.cc:729] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0081ea27154dcaac0b971d16e1c0d2, State: Initialized, Role: FOLLOWER
I20260811 03:01:34.617403 15323 ts_tablet_manager.cc:1397] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 03:01:34.617657 15322 ts_tablet_manager.cc:1428] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:34.617616 15319 consensus_queue.cc:260] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.618009 14735 heartbeater.cc:503] Master 127.8.203.126:42845 was elected leader, sending a full tablet report...
I20260811 03:01:34.618219 15286 heartbeater.cc:503] Master 127.8.203.126:42845 was elected leader, sending a full tablet report...
I20260811 03:01:34.618482 15319 ts_tablet_manager.cc:1428] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:34.620976 15323 raft_consensus.cc:348] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.621204 15323 raft_consensus.cc:374] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:34.621259 15323 raft_consensus.cc:729] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3678c5c9bc254255b5ee2ffbd44dc0b7, State: Initialized, Role: FOLLOWER
I20260811 03:01:34.621670 15323 consensus_queue.cc:260] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.622396 15017 heartbeater.cc:503] Master 127.8.203.126:42845 was elected leader, sending a full tablet report...
I20260811 03:01:34.622611 15323 ts_tablet_manager.cc:1428] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:34.700875 15336 raft_consensus.cc:482] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:34.701068 15336 raft_consensus.cc:504] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.702219 15336 leader_election.cc:288] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0df1e73442f2490a971a7ee4bf303e36 (127.8.203.65:40239), 3678c5c9bc254255b5ee2ffbd44dc0b7 (127.8.203.66:33151)
I20260811 03:01:34.705657 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.702451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40239 (local address 127.8.203.67:50969)
0811 03:01:34.702588 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.702604 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.702706 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.703019 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.703023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.703042 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:34.703154 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.703159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.703848 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.703851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.704618 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.704628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.705223 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.705227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.705362 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.705383 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.705451 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.705524 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 03:01:34.705657 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.702558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:50969 (local address 127.8.203.65:40239)
0811 03:01:34.702700 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.702705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.702722 (+ 17us) server_negotiation.cc:407] Connection header received
0811 03:01:34.702816 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.702820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.702886 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.702966 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:34.703260 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.703747 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.704737 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.705121 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.705223 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.705278 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.705476 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.705545 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 03:01:34.706239 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.702726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:33151 (local address 127.8.203.67:60451)
0811 03:01:34.703072 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.703083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:34.703192 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:34.703418 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:34.703421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:34.703436 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:34.703552 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.703557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.704230 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.704234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.705003 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:34.705012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.705701 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.705832 (+ 131us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:34.705941 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.705964 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 03:01:34.706044 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 03:01:34.706109 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":125,"threads_started":1}
I20260811 03:01:34.706261 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.702844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:60451 (local address 127.8.203.66:33151)
0811 03:01:34.702968 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.702972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.703162 (+ 190us) server_negotiation.cc:407] Connection header received
0811 03:01:34.703275 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.703278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.703325 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.703399 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:34.703647 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.704127 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.705119 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.705576 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.705683 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.705730 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.706079 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.706152 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:01:34.706288 14635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf78d56a22f40d5ad40675d8da2e623" candidate_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df1e73442f2490a971a7ee4bf303e36" is_pre_election: true
I20260811 03:01:34.706621 14635 raft_consensus.cc:2454] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d0081ea27154dcaac0b971d16e1c0d2 in term 0.
I20260811 03:01:34.707026 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf78d56a22f40d5ad40675d8da2e623" candidate_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" is_pre_election: true
I20260811 03:01:34.707088 15063 leader_election.cc:302] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df1e73442f2490a971a7ee4bf303e36, 9d0081ea27154dcaac0b971d16e1c0d2; no voters:
I20260811 03:01:34.707268 14924 raft_consensus.cc:2454] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d0081ea27154dcaac0b971d16e1c0d2 in term 0.
I20260811 03:01:34.707360 15336 raft_consensus.cc:2781] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:01:34.707448 15336 raft_consensus.cc:482] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:01:34.707497 15336 raft_consensus.cc:3037] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:34.708545 15336 raft_consensus.cc:504] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.708837 15336 leader_election.cc:288] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [CANDIDATE]: Term 1 election: Requested vote from peers 0df1e73442f2490a971a7ee4bf303e36 (127.8.203.65:40239), 3678c5c9bc254255b5ee2ffbd44dc0b7 (127.8.203.66:33151)
I20260811 03:01:34.709199 14924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf78d56a22f40d5ad40675d8da2e623" candidate_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7"
I20260811 03:01:34.709170 14635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf78d56a22f40d5ad40675d8da2e623" candidate_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0df1e73442f2490a971a7ee4bf303e36"
I20260811 03:01:34.709328 14635 raft_consensus.cc:3037] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:34.709342 14924 raft_consensus.cc:3037] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:34.710520 14635 raft_consensus.cc:2454] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d0081ea27154dcaac0b971d16e1c0d2 in term 1.
I20260811 03:01:34.710520 14924 raft_consensus.cc:2454] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d0081ea27154dcaac0b971d16e1c0d2 in term 1.
I20260811 03:01:34.710814 15063 leader_election.cc:302] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0df1e73442f2490a971a7ee4bf303e36, 9d0081ea27154dcaac0b971d16e1c0d2; no voters:
I20260811 03:01:34.710990 15336 raft_consensus.cc:2781] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:34.711249 15336 raft_consensus.cc:686] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 1 LEADER]: Becoming Leader. State: Replica: 9d0081ea27154dcaac0b971d16e1c0d2, State: Running, Role: LEADER
I20260811 03:01:34.711530 15336 consensus_queue.cc:237] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } }
I20260811 03:01:34.714661 14340 catalog_manager.cc:5495] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d0081ea27154dcaac0b971d16e1c0d2 (127.8.203.67). New cstate: current_term: 1 leader_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d0081ea27154dcaac0b971d16e1c0d2" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260811 03:01:34.743561 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.740407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.8.203.67:33721)
0811 03:01:34.740514 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:34.740518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:34.740529 (+ 11us) server_negotiation.cc:407] Connection header received
0811 03:01:34.740657 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:34.740659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:34.740709 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:34.740791 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:01:34.740977 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.741516 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.742316 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:34.742540 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:34.742627 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:34.742867 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:01:34.743265 (+ 398us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:01:34.743359 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:34.743411 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 03:01:34.743458 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 03:01:34.749115 14924 raft_consensus.cc:1261] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 1 FOLLOWER]: Refusing update from remote peer 9d0081ea27154dcaac0b971d16e1c0d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:34.749115 14635 raft_consensus.cc:1261] T 5bf78d56a22f40d5ad40675d8da2e623 P 0df1e73442f2490a971a7ee4bf303e36 [term 1 FOLLOWER]: Refusing update from remote peer 9d0081ea27154dcaac0b971d16e1c0d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:34.749820 15336 consensus_queue.cc:1035] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3678c5c9bc254255b5ee2ffbd44dc0b7" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 33151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:34.750022 15346 consensus_queue.cc:1035] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0df1e73442f2490a971a7ee4bf303e36" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:34.758054 15354 mvcc.cc:205] Tried to move back new op lower bound from 7317165239285456896 to 7317165239140978688. Current Snapshot: MvccSnapshot[applied={T|T < 7317165239285456896}]
I20260811 03:01:34.758843 15358 mvcc.cc:205] Tried to move back new op lower bound from 7317165239285456896 to 7317165239140978688. Current Snapshot: MvccSnapshot[applied={T|T < 7317165239285456896}]
I20260811 03:01:34.765648 14339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:34.765817 14339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:34.768232 14339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e72920f121ac4e57a1f78cda53d2ce98: Sending DeleteTablet for 3 replicas of tablet 5bf78d56a22f40d5ad40675d8da2e623
I20260811 03:01:34.768991 15149 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf78d56a22f40d5ad40675d8da2e623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:34 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:58902
I20260811 03:01:34.769145 14886 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf78d56a22f40d5ad40675d8da2e623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:34 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:33134
I20260811 03:01:34.769145 14590 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf78d56a22f40d5ad40675d8da2e623 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:34 UTC) from {username='kudu', principal='kudu/127.8.203.126@KRBTEST.COM'} at 127.0.0.1:48492
I20260811 03:01:34.769448 15374 tablet_replica.cc:330] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: stopping tablet replica
I20260811 03:01:34.769526 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 14460
I20260811 03:01:34.773715 15378 tablet_replica.cc:330] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: stopping tablet replica
I20260811 03:01:34.773806 15374 raft_consensus.cc:2229] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:01:34.774266 15374 raft_consensus.cc:2258] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:34.776579 15378 raft_consensus.cc:2229] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:34.776955 15378 raft_consensus.cc:2258] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:34.777815 15374 ts_tablet_manager.cc:1905] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:34.780237 15374 ts_tablet_manager.cc:1918] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:34.780323 15374 log.cc:1196] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/5bf78d56a22f40d5ad40675d8da2e623
I20260811 03:01:34.780617 15374 ts_tablet_manager.cc:1939] T 5bf78d56a22f40d5ad40675d8da2e623 P 9d0081ea27154dcaac0b971d16e1c0d2: Deleting consensus metadata
I20260811 03:01:34.781606 14312 catalog_manager.cc:4841] TS 9d0081ea27154dcaac0b971d16e1c0d2 (127.8.203.67:33721): tablet 5bf78d56a22f40d5ad40675d8da2e623 (table test-table [id=088ac8b667b74768904084560df16097]) successfully deleted
I20260811 03:01:34.785605 15378 ts_tablet_manager.cc:1905] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:34.787707 15378 ts_tablet_manager.cc:1918] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:34.787796 15378 log.cc:1196] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/5bf78d56a22f40d5ad40675d8da2e623
I20260811 03:01:34.788058 15378 ts_tablet_manager.cc:1939] T 5bf78d56a22f40d5ad40675d8da2e623 P 3678c5c9bc254255b5ee2ffbd44dc0b7: Deleting consensus metadata
W20260811 03:01:34.788261 14311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:40239 (error 108)
I20260811 03:01:34.788470 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 14740
I20260811 03:01:34.795499 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.793900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40239 (local address 127.0.0.1:48498)
0811 03:01:34.795320 (+ 1420us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.795404 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:40239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1316}
W20260811 03:01:34.796247 14311 catalog_manager.cc:4584] TS 0df1e73442f2490a971a7ee4bf303e36 (127.8.203.65:40239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bf78d56a22f40d5ad40675d8da2e623: Network error: Client connection negotiation failed: client connection to 127.8.203.65:40239: connect: Connection refused (error 111)
I20260811 03:01:34.800823 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15028
I20260811 03:01:34.806170 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:34.801015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:33151 (local address 127.0.0.1:33136)
0811 03:01:34.805995 (+ 4980us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:34.806070 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.66:33151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4880}
I20260811 03:01:34.812323 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 14284
2026-08-11T03:01:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2428 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T03:01:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:34Z Disabled control of system clock
I20260811 03:01:34.852480 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41065
--webserver_interface=127.8.203.126
--webserver_port=0
--builtin_ntp_servers=127.8.203.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:41065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:01:34.961728 15393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:34.961966 15393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:34.961998 15393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:34.965837 15393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:34.965886 15393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:34.965900 15393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:34.965919 15393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:34.965936 15393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:34.970569 15393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:41065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:34.971519 15393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:34.972800 15393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:34.980795 15404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:34.980815 15412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:34.980803 15407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:34.981061 15393 server_base.cc:1034] running on GCE node
I20260811 03:01:34.981594 15393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:34.982571 15393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:34.983716 15393 hybrid_clock.cc:648] HybridClock initialized: now 1786417294983689 us; error 39 us; skew 500 ppm
I20260811 03:01:34.985452 15393 webserver.cc:460] Webserver started at http://127.8.203.126:36529/ using document root <none> and password file <none>
I20260811 03:01:34.985988 15393 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:34.986048 15393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:34.986240 15393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:34.988262 15393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "a08cc6a355f1442c94171f9495d96382"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "9e286c5560cfa1ff0600c5cb17b868d0"
server_key_iv: "6c23fa2c1456f7d7d1cb03d47e72307a"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.988794 15393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "a08cc6a355f1442c94171f9495d96382"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "9e286c5560cfa1ff0600c5cb17b868d0"
server_key_iv: "6c23fa2c1456f7d7d1cb03d47e72307a"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.992225 15393 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 03:01:34.994490 15423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:34.995579 15393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:34.995682 15393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "a08cc6a355f1442c94171f9495d96382"
format_stamp: "Formatted at 2026-08-11 03:01:34 on dist-test-slave-gjw7"
server_key: "9e286c5560cfa1ff0600c5cb17b868d0"
server_key_iv: "6c23fa2c1456f7d7d1cb03d47e72307a"
server_key_version: "encryptionkey@0"
I20260811 03:01:34.995774 15393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:35.030252 15393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:35.030954 15393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:35.031147 15393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:35.038224 15393 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:41065
I20260811 03:01:35.038236 15532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:41065 every 8 connection(s)
I20260811 03:01:35.039165 15393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:35.042214 15534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:35.047595 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382: Bootstrap starting.
I20260811 03:01:35.048147 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15393
I20260811 03:01:35.048291 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:35.048552 9005 external_mini_cluster.cc:1428] Setting key b402467f4ae58bd52c2aefe13d9242fa
I20260811 03:01:35.050225 15534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:35.050931 15534 log.cc:824] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:35.053020 15534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382: No bootstrap required, opened a new log
I20260811 03:01:35.057148 15534 raft_consensus.cc:348] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08cc6a355f1442c94171f9495d96382" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41065 } }
I20260811 03:01:35.057423 15534 raft_consensus.cc:374] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:35.057495 15534 raft_consensus.cc:729] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08cc6a355f1442c94171f9495d96382, State: Initialized, Role: FOLLOWER
I20260811 03:01:35.057942 15534 consensus_queue.cc:260] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08cc6a355f1442c94171f9495d96382" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41065 } }
I20260811 03:01:35.058077 15534 raft_consensus.cc:388] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:35.058089 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.049816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50208 (local address 127.8.203.126:41065)
0811 03:01:35.050500 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.050516 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.050554 (+ 38us) server_negotiation.cc:407] Connection header received
0811 03:01:35.051218 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.051242 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.051694 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.052024 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:35.052396 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.053426 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.054624 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.055119 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.055259 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.056131 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:35.056137 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:35.056149 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:35.056541 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:35.056601 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:35.056623 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:35.056714 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.056862 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.057144 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":131,"threads_started":1}
I20260811 03:01:35.058159 15534 raft_consensus.cc:482] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:35.058261 15534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:35.059366 15534 raft_consensus.cc:504] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08cc6a355f1442c94171f9495d96382" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41065 } }
I20260811 03:01:35.059692 15534 leader_election.cc:302] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a08cc6a355f1442c94171f9495d96382; no voters:
I20260811 03:01:35.059968 15534 leader_election.cc:288] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:35.060082 15539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:35.060331 15539 raft_consensus.cc:686] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [term 1 LEADER]: Becoming Leader. State: Replica: a08cc6a355f1442c94171f9495d96382, State: Running, Role: LEADER
I20260811 03:01:35.060664 15539 consensus_queue.cc:237] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08cc6a355f1442c94171f9495d96382" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41065 } }
I20260811 03:01:35.061206 15534 sys_catalog.cc:564] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:35.061772 15539 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a08cc6a355f1442c94171f9495d96382. Latest consensus state: current_term: 1 leader_uuid: "a08cc6a355f1442c94171f9495d96382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08cc6a355f1442c94171f9495d96382" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41065 } } }
I20260811 03:01:35.061892 15539 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:35.062625 15541 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a08cc6a355f1442c94171f9495d96382" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08cc6a355f1442c94171f9495d96382" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41065 } } }
I20260811 03:01:35.062743 15541 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382 [sys.catalog]: This master's current role is: LEADER
W20260811 03:01:35.066097 15568 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:01:35.066169 15568 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:01:35.066226 15553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:35.066941 15553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:35.071931 15553 catalog_manager.cc:1347] Generated new cluster ID: e9b5e9df7a6f4c82b6af26a124de404a
I20260811 03:01:35.071995 15553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:35.103956 15553 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:35.104875 15553 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:35.110999 15553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382: Generated new TSK 0
I20260811 03:01:35.111644 15553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:35.116834 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.65:0
--local_ip_for_outbound_sockets=127.8.203.65
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41065
--builtin_ntp_servers=127.8.203.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 03:01:35.227141 15574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:35.227378 15574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:35.227420 15574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:35.231293 15574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:35.231334 15574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:35.231405 15574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.65
I20260811 03:01:35.235965 15574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.203.65
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.65
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:35.236917 15574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:35.238276 15574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:35.247268 15585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:35.247347 15587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:35.247329 15574 server_base.cc:1034] running on GCE node
W20260811 03:01:35.247274 15589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:35.247900 15574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:35.248451 15574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:35.249599 15574 hybrid_clock.cc:648] HybridClock initialized: now 1786417295249581 us; error 29 us; skew 500 ppm
I20260811 03:01:35.251271 15574 webserver.cc:460] Webserver started at http://127.8.203.65:39761/ using document root <none> and password file <none>
I20260811 03:01:35.251808 15574 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:35.251871 15574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:35.252059 15574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:35.254037 15574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data/instance:
uuid: "b27fecc619c84e2097abe940acdc1c7e"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "e8c505cd1d1e1d9f6f751613ffa0d36c"
server_key_iv: "d7c14f63efcc6765a9dba131490271bc"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.254549 15574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance:
uuid: "b27fecc619c84e2097abe940acdc1c7e"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "e8c505cd1d1e1d9f6f751613ffa0d36c"
server_key_iv: "d7c14f63efcc6765a9dba131490271bc"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.257969 15574 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:01:35.260152 15605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.261277 15574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:35.261427 15574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/wal
uuid: "b27fecc619c84e2097abe940acdc1c7e"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "e8c505cd1d1e1d9f6f751613ffa0d36c"
server_key_iv: "d7c14f63efcc6765a9dba131490271bc"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.261518 15574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:35.279588 15574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:35.280211 15574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:35.280377 15574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:35.280928 15574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:35.281857 15574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:35.281906 15574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.281960 15574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:35.281984 15574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.290078 15574 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.65:40931
I20260811 03:01:35.290110 15850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.65:40931 every 8 connection(s)
I20260811 03:01:35.290899 15574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/data/info.pb
I20260811 03:01:35.293416 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15574
I20260811 03:01:35.293538 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-0/wal/instance
I20260811 03:01:35.293841 9005 external_mini_cluster.cc:1428] Setting key c2ef2fe7373437b5455f3c39d58af946
I20260811 03:01:35.295646 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.66:0
--local_ip_for_outbound_sockets=127.8.203.66
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41065
--builtin_ntp_servers=127.8.203.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:01:35.299787 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.292537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.65:50627 (local address 127.8.203.126:41065)
0811 03:01:35.292672 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.292677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.293761 (+ 1084us) server_negotiation.cc:407] Connection header received
0811 03:01:35.295233 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.295236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.295289 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.295359 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:35.296300 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.296823 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.297984 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.298201 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.298290 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.299326 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:35.299329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:35.299331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:35.299344 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:35.299363 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:35.299371 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:35.299425 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.299586 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.299645 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 03:01:35.300590 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.292932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41065 (local address 127.8.203.65:50627)
0811 03:01:35.293620 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.293663 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.295036 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.295545 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.295552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.295636 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:35.296161 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.296175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.296952 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.296955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.297868 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.297879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.298344 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.298349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.298515 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.299063 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:35.299065 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:35.299107 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:35.299185 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:35.299436 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:35.299446 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:35.299452 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.299558 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.299821 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":110,"threads_started":1}
I20260811 03:01:35.301699 15851 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41065
I20260811 03:01:35.301941 15851 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:35.302544 15851 heartbeater.cc:511] Master 127.8.203.126:41065 requested a full tablet report, sending...
I20260811 03:01:35.304033 15452 ts_manager.cc:194] Registered new tserver with Master: b27fecc619c84e2097abe940acdc1c7e (127.8.203.65:40931)
I20260811 03:01:35.305351 15452 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.65:50627
W20260811 03:01:35.406534 15857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:35.406769 15857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:35.406818 15857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:35.410691 15857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:35.410737 15857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:35.410807 15857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.66
I20260811 03:01:35.415412 15857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.203.66
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.66
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:35.416401 15857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:35.417722 15857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:35.426810 15865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:35.426895 15857 server_base.cc:1034] running on GCE node
W20260811 03:01:35.426803 15872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:35.426803 15870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:35.427542 15857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:35.428215 15857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:35.429373 15857 hybrid_clock.cc:648] HybridClock initialized: now 1786417295429346 us; error 25 us; skew 500 ppm
I20260811 03:01:35.431111 15857 webserver.cc:460] Webserver started at http://127.8.203.66:43161/ using document root <none> and password file <none>
I20260811 03:01:35.431660 15857 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:35.431727 15857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:35.431929 15857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:35.433950 15857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data/instance:
uuid: "a669b1572a104a43ba96ede55d46492a"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "68af936baee60d5a76a5de93ba16c0b3"
server_key_iv: "00fd6eeb71fa6975658f12b755ea193b"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.434490 15857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance:
uuid: "a669b1572a104a43ba96ede55d46492a"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "68af936baee60d5a76a5de93ba16c0b3"
server_key_iv: "00fd6eeb71fa6975658f12b755ea193b"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.437978 15857 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:01:35.440183 15885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.441272 15857 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 03:01:35.441426 15857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal
uuid: "a669b1572a104a43ba96ede55d46492a"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "68af936baee60d5a76a5de93ba16c0b3"
server_key_iv: "00fd6eeb71fa6975658f12b755ea193b"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.441517 15857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:35.461457 15857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:35.462077 15857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:35.462236 15857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:35.462782 15857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:35.463701 15857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:35.463748 15857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.463801 15857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:35.463829 15857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.472072 15857 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.66:46383
I20260811 03:01:35.472098 16128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.66:46383 every 8 connection(s)
I20260811 03:01:35.472918 15857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/data/info.pb
I20260811 03:01:35.481565 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.474642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.66:57529 (local address 127.8.203.126:41065)
0811 03:01:35.474764 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.474767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.475760 (+ 993us) server_negotiation.cc:407] Connection header received
0811 03:01:35.477151 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.477155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.477216 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.477303 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:35.478200 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.478725 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.479833 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.480048 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.480138 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.481100 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:35.481104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:35.481105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:35.481117 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:35.481136 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:35.481144 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:35.481202 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.481361 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.481424 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:01:35.482363 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.474989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41065 (local address 127.8.203.66:57529)
0811 03:01:35.475612 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.475651 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.476936 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.477491 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.477498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.477582 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:35.478059 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.478072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.478855 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.478858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.479707 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.479717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.480192 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.480196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.480348 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.480851 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:35.480852 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:35.480892 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:35.480954 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:35.481212 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:35.481221 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:35.481228 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.481356 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.481615 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":124,"threads_started":1}
I20260811 03:01:35.482820 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 15857
I20260811 03:01:35.482945 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal/instance
I20260811 03:01:35.483278 9005 external_mini_cluster.cc:1428] Setting key 4285b94184cc27705c8ff4b9903cea99
I20260811 03:01:35.483449 16130 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41065
I20260811 03:01:35.483670 16130 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:35.484195 16130 heartbeater.cc:511] Master 127.8.203.126:41065 requested a full tablet report, sending...
I20260811 03:01:35.485204 15452 ts_manager.cc:194] Registered new tserver with Master: a669b1572a104a43ba96ede55d46492a (127.8.203.66:46383)
I20260811 03:01:35.485234 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.203.67:0
--local_ip_for_outbound_sockets=127.8.203.67
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41065
--builtin_ntp_servers=127.8.203.84:41957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 03:01:35.485882 15452 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.66:57529
W20260811 03:01:35.598726 16136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:35.598977 16136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:35.599035 16136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:35.602945 16136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:35.602996 16136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:35.603080 16136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.203.67
I20260811 03:01:35.607654 16136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.203.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.203.67
--webserver_port=0
--tserver_master_addrs=127.8.203.126:41065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.203.67
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:35.608613 16136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:35.609994 16136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:35.619560 16144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:35.619585 16145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:35.619709 16136 server_base.cc:1034] running on GCE node
W20260811 03:01:35.619603 16152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:35.620235 16136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:35.620831 16136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:35.621994 16136 hybrid_clock.cc:648] HybridClock initialized: now 1786417295621978 us; error 30 us; skew 500 ppm
I20260811 03:01:35.623850 16136 webserver.cc:460] Webserver started at http://127.8.203.67:33997/ using document root <none> and password file <none>
I20260811 03:01:35.624464 16136 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:35.624538 16136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:35.624760 16136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:35.626855 16136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data/instance:
uuid: "27b8b9b69c704018b9ceac1e032ed332"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "0521bb38fd4d3c1c9b19c5422e3742c7"
server_key_iv: "22ea7311b8c0ee589ef618b4826da411"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.627424 16136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance:
uuid: "27b8b9b69c704018b9ceac1e032ed332"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "0521bb38fd4d3c1c9b19c5422e3742c7"
server_key_iv: "22ea7311b8c0ee589ef618b4826da411"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.631047 16136 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 03:01:35.633440 16162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.634590 16136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 03:01:35.634706 16136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal
uuid: "27b8b9b69c704018b9ceac1e032ed332"
format_stamp: "Formatted at 2026-08-11 03:01:35 on dist-test-slave-gjw7"
server_key: "0521bb38fd4d3c1c9b19c5422e3742c7"
server_key_iv: "22ea7311b8c0ee589ef618b4826da411"
server_key_version: "encryptionkey@0"
I20260811 03:01:35.634806 16136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:35.654292 16136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:35.654887 16136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:35.655058 16136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:35.655617 16136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 03:01:35.656553 16136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 03:01:35.656605 16136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.656662 16136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 03:01:35.656697 16136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:35.665751 16136 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.67:44411
I20260811 03:01:35.665802 16406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.67:44411 every 8 connection(s)
I20260811 03:01:35.666626 16136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/data/info.pb
I20260811 03:01:35.671763 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 16136
I20260811 03:01:35.671880 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal/instance
I20260811 03:01:35.672149 9005 external_mini_cluster.cc:1428] Setting key 2f0b9112d7671636b133ef68041d68ed
I20260811 03:01:35.675879 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.668284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:35829 (local address 127.8.203.126:41065)
0811 03:01:35.668414 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.668417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.669452 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 03:01:35.670812 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.670816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.670878 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.670954 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:35.671924 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.672550 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.674146 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.674448 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.674530 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.675502 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:35.675505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:35.675507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:35.675519 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:35.675538 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:35.675546 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:35.675598 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.675721 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.675770 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:01:35.676712 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.668651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:41065 (local address 127.8.203.67:35829)
0811 03:01:35.669265 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.669355 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.670600 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.671135 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.671141 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.671224 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:01:35.671766 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.671782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.672868 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.672871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.674016 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.674028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.674580 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.674583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.674739 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.675260 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 03:01:35.675261 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:01:35.675301 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 03:01:35.675365 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:01:35.675609 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:01:35.675614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:01:35.675617 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.675720 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.675954 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":115,"threads_started":1}
I20260811 03:01:35.677732 16411 heartbeater.cc:347] Connected to a master server at 127.8.203.126:41065
I20260811 03:01:35.677951 16411 heartbeater.cc:464] Registering TS with master...
I20260811 03:01:35.678457 16411 heartbeater.cc:511] Master 127.8.203.126:41065 requested a full tablet report, sending...
I20260811 03:01:35.679421 15452 ts_manager.cc:194] Registered new tserver with Master: 27b8b9b69c704018b9ceac1e032ed332 (127.8.203.67:44411)
I20260811 03:01:35.679960 15452 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.203.67:35829
I20260811 03:01:35.685803 9005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 03:01:35.690992 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.687753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50218 (local address 127.8.203.126:41065)
0811 03:01:35.687881 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.687886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.688062 (+ 176us) server_negotiation.cc:407] Connection header received
0811 03:01:35.688159 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.688161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.688199 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.688257 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:35.688466 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.688933 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.689748 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.689947 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.690015 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.690659 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:35.690662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:35.690663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:35.690675 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:35.690693 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:35.690700 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:35.690746 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.690818 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.690865 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 03:01:35.693421 15452 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 03:01:35.696244 15452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 03:01:35.709887 16430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.704039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.67:44411 (local address 127.0.0.1:53788)
0811 03:01:35.705076 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.705127 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.705251 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.705980 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.705983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.706000 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:35.706168 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.706175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.707187 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.707189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.708365 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.708376 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.709433 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.709436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.709512 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.709538 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.709621 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.709733 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":30,"spinlock_wait_cycles":19712,"thread_start_us":55,"threads_started":1}
I20260811 03:01:35.709961 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.704179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.8.203.67:44411)
0811 03:01:35.704522 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.704532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.705217 (+ 685us) server_negotiation.cc:407] Connection header received
0811 03:01:35.705457 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.705463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.705673 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.705795 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:35.706289 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.707078 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.708497 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.709118 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.709177 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.709336 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.709669 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.709761 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":143,"threads_started":1}
I20260811 03:01:35.710374 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.704716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:46383 (local address 127.0.0.1:52970)
0811 03:01:35.705678 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.705691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.705772 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.706266 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.706269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.706287 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:35.706363 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.706378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.707620 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.707624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.708710 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.708720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.709945 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.709948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.710085 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.710109 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.710173 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.710241 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":4,"thread_start_us":106,"threads_started":1}
I20260811 03:01:35.710660 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.704705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.8.203.65:40931)
0811 03:01:35.705560 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.705571 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.705594 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:01:35.705663 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.705667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.705823 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.705964 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:35.706295 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.707458 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.708794 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.709599 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.709699 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.709873 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.710415 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.710512 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":121,"threads_started":1}
I20260811 03:01:35.710552 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.704214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40931 (local address 127.0.0.1:43500)
0811 03:01:35.705076 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.705127 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.705251 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.705964 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.705970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.705992 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:35.706165 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.706172 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.707583 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.707587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.708676 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.708686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.709940 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.709944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.710265 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.710287 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.710369 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.710432 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":235,"spinlock_wait_cycles":3200,"thread_start_us":102,"threads_started":1}
I20260811 03:01:35.710918 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.705024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.8.203.66:46383)
0811 03:01:35.705413 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.705427 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.705865 (+ 438us) server_negotiation.cc:407] Connection header received
0811 03:01:35.705949 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.705953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.706129 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.706278 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:35.706474 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.707536 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.708849 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.709599 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.709699 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.709889 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.710650 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.710754 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":88,"threads_started":1}
I20260811 03:01:35.713102 16273 tablet_service.cc:1467] Processing CreateTablet for tablet 486cc79ae38046fd8a9aaa1b9faefdc9 (DEFAULT_TABLE table=test-table [id=51ba0c4d064943eb8a031eb86d6c567e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:35.713572 15712 tablet_service.cc:1467] Processing CreateTablet for tablet 486cc79ae38046fd8a9aaa1b9faefdc9 (DEFAULT_TABLE table=test-table [id=51ba0c4d064943eb8a031eb86d6c567e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:35.713809 15987 tablet_service.cc:1467] Processing CreateTablet for tablet 486cc79ae38046fd8a9aaa1b9faefdc9 (DEFAULT_TABLE table=test-table [id=51ba0c4d064943eb8a031eb86d6c567e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 03:01:35.714278 16273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 486cc79ae38046fd8a9aaa1b9faefdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:35.714465 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 486cc79ae38046fd8a9aaa1b9faefdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:35.714761 15987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 486cc79ae38046fd8a9aaa1b9faefdc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:35.720847 16443 tablet_bootstrap.cc:492] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: Bootstrap starting.
I20260811 03:01:35.721956 16446 tablet_bootstrap.cc:492] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Bootstrap starting.
I20260811 03:01:35.723204 16443 tablet_bootstrap.cc:654] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:35.721462 16444 tablet_bootstrap.cc:492] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Bootstrap starting.
I20260811 03:01:35.724004 16446 tablet_bootstrap.cc:654] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:35.724104 16443 log.cc:824] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:35.724764 16446 log.cc:824] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:35.725206 16444 tablet_bootstrap.cc:654] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:35.726032 16444 log.cc:824] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:35.726548 16443 tablet_bootstrap.cc:492] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: No bootstrap required, opened a new log
I20260811 03:01:35.726749 16443 ts_tablet_manager.cc:1397] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 03:01:35.726830 16446 tablet_bootstrap.cc:492] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: No bootstrap required, opened a new log
I20260811 03:01:35.727007 16446 ts_tablet_manager.cc:1397] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 03:01:35.727708 16444 tablet_bootstrap.cc:492] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: No bootstrap required, opened a new log
I20260811 03:01:35.727882 16444 ts_tablet_manager.cc:1397] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260811 03:01:35.730891 16444 raft_consensus.cc:348] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.731105 16444 raft_consensus.cc:374] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:35.731158 16444 raft_consensus.cc:729] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a669b1572a104a43ba96ede55d46492a, State: Initialized, Role: FOLLOWER
I20260811 03:01:35.730976 16446 raft_consensus.cc:348] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.730976 16443 raft_consensus.cc:348] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.731212 16443 raft_consensus.cc:374] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:35.731212 16446 raft_consensus.cc:374] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:35.731269 16446 raft_consensus.cc:729] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27b8b9b69c704018b9ceac1e032ed332, State: Initialized, Role: FOLLOWER
I20260811 03:01:35.731279 16443 raft_consensus.cc:729] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27fecc619c84e2097abe940acdc1c7e, State: Initialized, Role: FOLLOWER
I20260811 03:01:35.731611 16444 consensus_queue.cc:260] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.731693 16446 consensus_queue.cc:260] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.731693 16443 consensus_queue.cc:260] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.732219 16130 heartbeater.cc:503] Master 127.8.203.126:41065 was elected leader, sending a full tablet report...
I20260811 03:01:35.732582 16444 ts_tablet_manager.cc:1428] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 03:01:35.732605 16446 ts_tablet_manager.cc:1428] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 03:01:35.732605 16443 ts_tablet_manager.cc:1428] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 03:01:35.732919 16411 heartbeater.cc:503] Master 127.8.203.126:41065 was elected leader, sending a full tablet report...
I20260811 03:01:35.732919 15851 heartbeater.cc:503] Master 127.8.203.126:41065 was elected leader, sending a full tablet report...
W20260811 03:01:35.791996 15854 tablet.cc:2369] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 03:01:35.917522 16412 tablet.cc:2369] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:01:35.950592 16454 raft_consensus.cc:482] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 03:01:35.950778 16454 raft_consensus.cc:504] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.951845 16454 leader_election.cc:288] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b27fecc619c84e2097abe940acdc1c7e (127.8.203.65:40931), a669b1572a104a43ba96ede55d46492a (127.8.203.66:46383)
I20260811 03:01:35.955224 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.952082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40931 (local address 127.8.203.67:39945)
0811 03:01:35.952207 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.952223 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.952328 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.952574 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.952577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.952599 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:35.952706 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.952711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.953446 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.953449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.954215 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.954224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.954830 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.954842 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.954936 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.954955 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.955019 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.955089 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 03:01:35.955224 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.952195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:39945 (local address 127.8.203.65:40931)
0811 03:01:35.952325 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.952330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.952361 (+ 31us) server_negotiation.cc:407] Connection header received
0811 03:01:35.952422 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.952425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.952474 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.952545 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:35.952812 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.953345 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.954348 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.954722 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.954825 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.954878 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.955044 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.955111 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:01:35.955603 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.952302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.203.67:50077 (local address 127.8.203.66:46383)
0811 03:01:35.952412 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.952416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.952642 (+ 226us) server_negotiation.cc:407] Connection header received
0811 03:01:35.952741 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.952743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.952791 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.952853 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 03:01:35.953136 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.953667 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.954656 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.955037 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.955141 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.955187 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.955399 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.955478 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 03:01:35.955873 15760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486cc79ae38046fd8a9aaa1b9faefdc9" candidate_uuid: "27b8b9b69c704018b9ceac1e032ed332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27fecc619c84e2097abe940acdc1c7e" is_pre_election: true
I20260811 03:01:35.955956 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.952082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.66:46383 (local address 127.8.203.67:50077)
0811 03:01:35.952547 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:35.952557 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 03:01:35.952655 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:01:35.952887 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:01:35.952890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:01:35.952906 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 03:01:35.953021 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.953026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.953774 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.953778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.954529 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:01:35.954537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.955154 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.955158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:01:35.955251 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.955273 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 03:01:35.955744 (+ 471us) client_negotiation.cc:240] Negotiation successful
0811 03:01:35.955818 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":42,"spinlock_wait_cycles":1792,"thread_start_us":121,"threads_started":1}
I20260811 03:01:35.956189 15760 raft_consensus.cc:2454] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27b8b9b69c704018b9ceac1e032ed332 in term 0.
I20260811 03:01:35.956568 16031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486cc79ae38046fd8a9aaa1b9faefdc9" candidate_uuid: "27b8b9b69c704018b9ceac1e032ed332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a669b1572a104a43ba96ede55d46492a" is_pre_election: true
I20260811 03:01:35.956687 16166 leader_election.cc:302] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27b8b9b69c704018b9ceac1e032ed332, b27fecc619c84e2097abe940acdc1c7e; no voters:
I20260811 03:01:35.956863 16031 raft_consensus.cc:2454] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27b8b9b69c704018b9ceac1e032ed332 in term 0.
I20260811 03:01:35.956971 16454 raft_consensus.cc:2781] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 03:01:35.957070 16454 raft_consensus.cc:482] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 03:01:35.957123 16454 raft_consensus.cc:3037] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:35.958201 16454 raft_consensus.cc:504] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.958515 16454 leader_election.cc:288] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [CANDIDATE]: Term 1 election: Requested vote from peers b27fecc619c84e2097abe940acdc1c7e (127.8.203.65:40931), a669b1572a104a43ba96ede55d46492a (127.8.203.66:46383)
I20260811 03:01:35.958824 15760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486cc79ae38046fd8a9aaa1b9faefdc9" candidate_uuid: "27b8b9b69c704018b9ceac1e032ed332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27fecc619c84e2097abe940acdc1c7e"
I20260811 03:01:35.958935 16031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "486cc79ae38046fd8a9aaa1b9faefdc9" candidate_uuid: "27b8b9b69c704018b9ceac1e032ed332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a669b1572a104a43ba96ede55d46492a"
I20260811 03:01:35.958961 15760 raft_consensus.cc:3037] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:35.959050 16031 raft_consensus.cc:3037] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:35.960197 15760 raft_consensus.cc:2454] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27b8b9b69c704018b9ceac1e032ed332 in term 1.
I20260811 03:01:35.960197 16031 raft_consensus.cc:2454] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27b8b9b69c704018b9ceac1e032ed332 in term 1.
I20260811 03:01:35.960497 16166 leader_election.cc:302] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 27b8b9b69c704018b9ceac1e032ed332, a669b1572a104a43ba96ede55d46492a, b27fecc619c84e2097abe940acdc1c7e; no voters:
I20260811 03:01:35.960662 16454 raft_consensus.cc:2781] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:35.960893 16454 raft_consensus.cc:686] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 1 LEADER]: Becoming Leader. State: Replica: 27b8b9b69c704018b9ceac1e032ed332, State: Running, Role: LEADER
I20260811 03:01:35.961221 16454 consensus_queue.cc:237] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } }
I20260811 03:01:35.964471 15451 catalog_manager.cc:5495] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27b8b9b69c704018b9ceac1e032ed332 (127.8.203.67). New cstate: current_term: 1 leader_uuid: "27b8b9b69c704018b9ceac1e032ed332" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27b8b9b69c704018b9ceac1e032ed332" member_type: VOTER last_known_addr { host: "127.8.203.67" port: 44411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 } health_report { overall_health: UNKNOWN } } }
W20260811 03:01:35.973971 16131 tablet.cc:2369] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:01:35.999445 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:35.996060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.8.203.67:44411)
0811 03:01:35.996194 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:35.996199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:35.996212 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:01:35.996287 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:35.996289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:35.996340 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:35.996431 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:01:35.996664 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.997155 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.998058 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:35.998268 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:35.998354 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:35.998641 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:01:35.999073 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:01:35.999181 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:35.999246 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 03:01:35.999297 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 03:01:36.004966 16031 raft_consensus.cc:1261] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 1 FOLLOWER]: Refusing update from remote peer 27b8b9b69c704018b9ceac1e032ed332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:36.004966 15760 raft_consensus.cc:1261] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 1 FOLLOWER]: Refusing update from remote peer 27b8b9b69c704018b9ceac1e032ed332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:01:36.005649 16454 consensus_queue.cc:1035] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a669b1572a104a43ba96ede55d46492a" member_type: VOTER last_known_addr { host: "127.8.203.66" port: 46383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:36.005820 16458 consensus_queue.cc:1035] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b27fecc619c84e2097abe940acdc1c7e" member_type: VOTER last_known_addr { host: "127.8.203.65" port: 40931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:01:36.014204 16466 mvcc.cc:205] Tried to move back new op lower bound from 7317165244429643776 to 7317165244259770368. Current Snapshot: MvccSnapshot[applied={T|T < 7317165244429643776 or (T in {7317165244429643776})}]
I20260811 03:01:36.014765 16469 mvcc.cc:205] Tried to move back new op lower bound from 7317165244429643776 to 7317165244259770368. Current Snapshot: MvccSnapshot[applied={T|T < 7317165244429643776}]
I20260811 03:01:36.019738 15450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:36.019912 15450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 03:01:36.022285 15450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a08cc6a355f1442c94171f9495d96382: Sending DeleteTablet for 3 replicas of tablet 486cc79ae38046fd8a9aaa1b9faefdc9
I20260811 03:01:36.023036 15987 tablet_service.cc:1514] Processing DeleteTablet for tablet 486cc79ae38046fd8a9aaa1b9faefdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:36 UTC) from {username='slave'} at 127.0.0.1:52970
I20260811 03:01:36.023030 15712 tablet_service.cc:1514] Processing DeleteTablet for tablet 486cc79ae38046fd8a9aaa1b9faefdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:36 UTC) from {username='slave'} at 127.0.0.1:43500
I20260811 03:01:36.023039 16273 tablet_service.cc:1514] Processing DeleteTablet for tablet 486cc79ae38046fd8a9aaa1b9faefdc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 03:01:36 UTC) from {username='slave'} at 127.0.0.1:53788
I20260811 03:01:36.023511 16489 tablet_replica.cc:330] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e: stopping tablet replica
I20260811 03:01:36.023818 16489 raft_consensus.cc:2229] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:36.024108 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15574
I20260811 03:01:36.024142 16489 raft_consensus.cc:2258] T 486cc79ae38046fd8a9aaa1b9faefdc9 P b27fecc619c84e2097abe940acdc1c7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:36.024487 16492 tablet_replica.cc:330] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: stopping tablet replica
I20260811 03:01:36.024762 16492 raft_consensus.cc:2229] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 03:01:36.025048 16492 raft_consensus.cc:2258] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:36.027567 16494 tablet_replica.cc:330] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: stopping tablet replica
I20260811 03:01:36.027828 16494 raft_consensus.cc:2229] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 1 LEADER]: Raft consensus shutting down.
I20260811 03:01:36.028141 16494 raft_consensus.cc:2258] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 03:01:36.029270 16492 ts_tablet_manager.cc:1905] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:36.031992 16492 ts_tablet_manager.cc:1918] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:36.032089 16492 log.cc:1196] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-1/wal/wals/486cc79ae38046fd8a9aaa1b9faefdc9
I20260811 03:01:36.032378 16492 ts_tablet_manager.cc:1939] T 486cc79ae38046fd8a9aaa1b9faefdc9 P a669b1572a104a43ba96ede55d46492a: Deleting consensus metadata
I20260811 03:01:36.033722 15426 catalog_manager.cc:4841] TS a669b1572a104a43ba96ede55d46492a (127.8.203.66:46383): tablet 486cc79ae38046fd8a9aaa1b9faefdc9 (table test-table [id=51ba0c4d064943eb8a031eb86d6c567e]) successfully deleted
I20260811 03:01:36.037667 16494 ts_tablet_manager.cc:1905] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 03:01:36.040304 16494 ts_tablet_manager.cc:1918] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 03:01:36.040400 16494 log.cc:1196] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Deleting WAL directory at /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786417039021389-9005-0/minicluster-data/ts-2/wal/wals/486cc79ae38046fd8a9aaa1b9faefdc9
I20260811 03:01:36.040673 16494 ts_tablet_manager.cc:1939] T 486cc79ae38046fd8a9aaa1b9faefdc9 P 27b8b9b69c704018b9ceac1e032ed332: Deleting consensus metadata
I20260811 03:01:36.046293 15427 catalog_manager.cc:4841] TS 27b8b9b69c704018b9ceac1e032ed332 (127.8.203.67:44411): tablet 486cc79ae38046fd8a9aaa1b9faefdc9 (table test-table [id=51ba0c4d064943eb8a031eb86d6c567e]) successfully deleted
W20260811 03:01:36.053584 15425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.203.65:40931 (error 108)
I20260811 03:01:36.054487 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15857
I20260811 03:01:36.054738 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:36.054443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.65:40931 (local address 127.0.0.1:43504)
0811 03:01:36.054593 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 03:01:36.054661 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.65:40931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 03:01:36.069531 15425 catalog_manager.cc:4584] TS b27fecc619c84e2097abe940acdc1c7e (127.8.203.65:40931): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 486cc79ae38046fd8a9aaa1b9faefdc9: Network error: Client connection negotiation failed: client connection to 127.8.203.65:40931: connect: Connection refused (error 111)
I20260811 03:01:36.081614 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 16136
I20260811 03:01:36.096328 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 15393
2026-08-11T03:01:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1285 ms)
[----------] 6 tests from EncryptionPolicyTest (10909 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:36 dist-test-slave-gjw7 krb5kdc[16510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:36 dist-test-slave-gjw7 krb5kdc[16510](info): set up 2 sockets
Aug 11 03:01:36 dist-test-slave-gjw7 krb5kdc[16510](info): commencing operation
krb5kdc: starting...
W20260811 03:01:37.292304 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:37 dist-test-slave-gjw7 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:37.405216 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:37.514014 16526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:37.514266 16526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:37.514321 16526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:37.518234 16526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:37.518290 16526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:37.518317 16526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:37.518338 16526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:37.518359 16526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:37.523437 16526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:37.524459 16526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:37.525501 16526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:01:37.533631 16526 server_base.cc:1034] running on GCE node
W20260811 03:01:37.533597 16534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:37.533613 16538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:37.533597 16533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:37.534340 16526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:37.535382 16526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:37.536527 16526 hybrid_clock.cc:648] HybridClock initialized: now 1786417297536506 us; error 37 us; skew 500 ppm
Aug 11 03:01:37 dist-test-slave-gjw7 krb5kdc[16510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417297, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:37.539069 16526 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:37.539881 16526 webserver.cc:460] Webserver started at http://127.0.0.1:41021/ using document root <none> and password file <none>
I20260811 03:01:37.540426 16526 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:37.540491 16526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:37.540696 16526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:37.542721 16526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "51623bfe4c21401997f0f76deb0a1099"
format_stamp: "Formatted at 2026-08-11 03:01:37 on dist-test-slave-gjw7"
server_key: "10a3787634b41a79e1c29aa0d72c72f4"
server_key_iv: "86993aa7c3e24ae8486ffb62eadea765"
server_key_version: "encryptionkey@0"
I20260811 03:01:37.543267 16526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "51623bfe4c21401997f0f76deb0a1099"
format_stamp: "Formatted at 2026-08-11 03:01:37 on dist-test-slave-gjw7"
server_key: "10a3787634b41a79e1c29aa0d72c72f4"
server_key_iv: "86993aa7c3e24ae8486ffb62eadea765"
server_key_version: "encryptionkey@0"
I20260811 03:01:37.546795 16526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 03:01:37.549052 16548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:37.550215 16526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 03:01:37.550323 16526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "51623bfe4c21401997f0f76deb0a1099"
format_stamp: "Formatted at 2026-08-11 03:01:37 on dist-test-slave-gjw7"
server_key: "10a3787634b41a79e1c29aa0d72c72f4"
server_key_iv: "86993aa7c3e24ae8486ffb62eadea765"
server_key_version: "encryptionkey@0"
I20260811 03:01:37.550413 16526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:37.562945 16526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:37.565522 16526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:37.565681 16526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:37.572227 16526 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42245
I20260811 03:01:37.572247 16638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42245 every 8 connection(s)
I20260811 03:01:37.573119 16526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:37.576022 16640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:37.580229 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 16526
I20260811 03:01:37.580345 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:37.580608 9005 external_mini_cluster.cc:1428] Setting key 3a89525c1e9e3053cbe8b08afd0658de
I20260811 03:01:37.581676 16640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099: Bootstrap starting.
I20260811 03:01:37.583894 16640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:37.584753 16640 log.cc:824] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:37.586805 16640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099: No bootstrap required, opened a new log
Aug 11 03:01:37 dist-test-slave-gjw7 krb5kdc[16510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:01:37.590703 16640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51623bfe4c21401997f0f76deb0a1099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42245 } }
I20260811 03:01:37.590920 16640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:37.590979 16640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51623bfe4c21401997f0f76deb0a1099, State: Initialized, Role: FOLLOWER
I20260811 03:01:37.591413 16640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51623bfe4c21401997f0f76deb0a1099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42245 } }
I20260811 03:01:37.591550 16640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:37.591614 16640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:37.591712 16640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:37.592522 16640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51623bfe4c21401997f0f76deb0a1099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42245 } }
I20260811 03:01:37.592841 16640 leader_election.cc:302] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51623bfe4c21401997f0f76deb0a1099; no voters:
I20260811 03:01:37.593122 16640 leader_election.cc:288] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:37.593248 16651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:37.593497 16651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [term 1 LEADER]: Becoming Leader. State: Replica: 51623bfe4c21401997f0f76deb0a1099, State: Running, Role: LEADER
I20260811 03:01:37.593998 16651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51623bfe4c21401997f0f76deb0a1099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42245 } }
I20260811 03:01:37.594419 16640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:37.594480 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:37.581965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42465 (local address 127.0.0.1:42245)
0811 03:01:37.582344 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:37.582360 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:37.582395 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:01:37.583018 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:37.583034 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:37.583361 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:37.583625 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:37.584551 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:37.585614 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:37.586707 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:37.587178 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:37.587288 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:37.589921 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:37.589927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:37.589940 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:37.589967 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:37.591537 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:37.592047 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:37.592053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:37.592057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:37.592164 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:37.592484 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:37.592487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:37.592488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:37.592903 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:37.593186 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:37.593454 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 03:01:37.593666 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260811 03:01:37.595182 16651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51623bfe4c21401997f0f76deb0a1099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51623bfe4c21401997f0f76deb0a1099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42245 } } }
I20260811 03:01:37.595305 16651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:37.595490 16651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51623bfe4c21401997f0f76deb0a1099. Latest consensus state: current_term: 1 leader_uuid: "51623bfe4c21401997f0f76deb0a1099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51623bfe4c21401997f0f76deb0a1099" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42245 } } }
I20260811 03:01:37.595578 16651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:37.596859 16671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:37.597620 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:37.600028 16671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:37.607280 16671 catalog_manager.cc:1347] Generated new cluster ID: 1d89bca803ab4477995963968ec8a705
I20260811 03:01:37.607326 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:37.599517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36001 (local address 127.0.0.1:42245)
0811 03:01:37.599625 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:37.599629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:37.599822 (+ 193us) server_negotiation.cc:407] Connection header received
0811 03:01:37.599921 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:37.599924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:37.599964 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:37.600025 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:37.600653 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:37.601165 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:37.601976 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:37.602336 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:37.602446 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:37.604008 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:37.604012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:37.604015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:37.604040 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:37.605675 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:37.606131 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:37.606134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:37.606135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:37.606213 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:37.606527 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:37.606530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:37.606533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:37.606698 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:37.606825 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:37.607083 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 03:01:37.607172 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 03:01:37.607345 16671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:37.637825 16671 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:37.638788 16671 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:37.645447 16671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51623bfe4c21401997f0f76deb0a1099: Generated new TSK 0
I20260811 03:01:37.646104 16671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:37.650902 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 16526
W20260811 03:01:37.663960 16499 connection.cc:535] client connection to 127.0.0.1:42245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42245): Connection reset by peer (error 104)
2026-08-11T03:01:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:37 dist-test-slave-gjw7 krb5kdc[16700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:37 dist-test-slave-gjw7 krb5kdc[16700](info): set up 2 sockets
Aug 11 03:01:37 dist-test-slave-gjw7 krb5kdc[16700](info): commencing operation
krb5kdc: starting...
W20260811 03:01:38.854002 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:38 dist-test-slave-gjw7 krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:38.976542 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:39.096050 16723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:39.096293 16723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:39.096335 16723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:39.100229 16723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:39.100278 16723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:39.100298 16723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:39.100328 16723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:39.100355 16723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:39.105401 16723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:39.106371 16723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:39.107378 16723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:39.115511 16730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:39.115487 16731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:39.115504 16733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:39.115739 16723 server_base.cc:1034] running on GCE node
I20260811 03:01:39.116297 16723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:39.117378 16723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:39.118532 16723 hybrid_clock.cc:648] HybridClock initialized: now 1786417299118513 us; error 30 us; skew 500 ppm
Aug 11 03:01:39 dist-test-slave-gjw7 krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417299, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:39.121089 16723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:39.121935 16723 webserver.cc:460] Webserver started at http://127.0.0.1:37623/ using document root <none> and password file <none>
I20260811 03:01:39.122453 16723 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:39.122519 16723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:39.122720 16723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:39.124694 16723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "b7e036e13f684264a9795615d5ca32de"
format_stamp: "Formatted at 2026-08-11 03:01:39 on dist-test-slave-gjw7"
server_key: "fa7645ee5d9aaef2e1ce141d104f6f5e"
server_key_iv: "d205394ea76c4bb916315014d3d829ad"
server_key_version: "encryptionkey@0"
I20260811 03:01:39.125249 16723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "b7e036e13f684264a9795615d5ca32de"
format_stamp: "Formatted at 2026-08-11 03:01:39 on dist-test-slave-gjw7"
server_key: "fa7645ee5d9aaef2e1ce141d104f6f5e"
server_key_iv: "d205394ea76c4bb916315014d3d829ad"
server_key_version: "encryptionkey@0"
I20260811 03:01:39.128742 16723 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:01:39.131063 16745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:39.132181 16723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:39.132292 16723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "b7e036e13f684264a9795615d5ca32de"
format_stamp: "Formatted at 2026-08-11 03:01:39 on dist-test-slave-gjw7"
server_key: "fa7645ee5d9aaef2e1ce141d104f6f5e"
server_key_iv: "d205394ea76c4bb916315014d3d829ad"
server_key_version: "encryptionkey@0"
I20260811 03:01:39.132380 16723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:39.167117 16723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:39.169793 16723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:39.169979 16723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:39.176783 16723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42711
I20260811 03:01:39.176805 16855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42711 every 8 connection(s)
I20260811 03:01:39.177719 16723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:39.180481 16856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:39.181037 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 16723
I20260811 03:01:39.181166 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:39.181504 9005 external_mini_cluster.cc:1428] Setting key d05c6fc477b084d8cbe43e373a654574
I20260811 03:01:39.186425 16856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de: Bootstrap starting.
I20260811 03:01:39.188851 16856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:39.189677 16856 log.cc:824] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:39.189991 16861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:39.182725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33153 (local address 127.0.0.1:42711)
0811 03:01:39.183124 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:39.183138 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:39.183173 (+ 35us) server_negotiation.cc:407] Connection header received
0811 03:01:39.183801 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:39.183818 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:39.184149 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:39.184412 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:39.184760 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:39.185773 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:39.186910 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:39.187361 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:39.187471 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:39.188337 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:39.188342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:39.188355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:39.188719 (+ 364us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:39.188774 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:39.188796 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:39.188902 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:39.189084 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 03:01:39.189217 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":101,"threads_started":1}
I20260811 03:01:39.191651 16856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de: No bootstrap required, opened a new log
I20260811 03:01:39.192778 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:39.195487 16856 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e036e13f684264a9795615d5ca32de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260811 03:01:39.195709 16856 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:39.195765 16856 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e036e13f684264a9795615d5ca32de, State: Initialized, Role: FOLLOWER
I20260811 03:01:39.196238 16856 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e036e13f684264a9795615d5ca32de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260811 03:01:39.196437 16856 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:39.196503 16856 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:39.196594 16856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:39.197593 16856 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e036e13f684264a9795615d5ca32de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260811 03:01:39.197954 16856 leader_election.cc:302] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7e036e13f684264a9795615d5ca32de; no voters:
I20260811 03:01:39.198290 16856 leader_election.cc:288] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:39.198382 16878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:39.198535 16861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:39.194669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38393 (local address 127.0.0.1:42711)
0811 03:01:39.194772 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:39.194775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:39.194895 (+ 120us) server_negotiation.cc:407] Connection header received
0811 03:01:39.195014 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:39.195016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:39.195051 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:39.195102 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:39.195355 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:39.195878 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:39.196940 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:39.197175 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:39.197251 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:39.198053 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:39.198056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:39.198059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:39.198073 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:39.198100 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:39.198112 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:39.198182 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:39.198357 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 03:01:39.198393 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:01:39.198660 16878 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [term 1 LEADER]: Becoming Leader. State: Replica: b7e036e13f684264a9795615d5ca32de, State: Running, Role: LEADER
I20260811 03:01:39.198988 16878 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e036e13f684264a9795615d5ca32de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } }
I20260811 03:01:39.199591 16856 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:39.200067 16878 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7e036e13f684264a9795615d5ca32de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e036e13f684264a9795615d5ca32de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } } }
I20260811 03:01:39.200243 16878 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:39.200469 16878 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e036e13f684264a9795615d5ca32de. Latest consensus state: current_term: 1 leader_uuid: "b7e036e13f684264a9795615d5ca32de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e036e13f684264a9795615d5ca32de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42711 } } }
I20260811 03:01:39.200555 16878 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:39.200839 16901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:39.204195 16901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:39.209429 16901 catalog_manager.cc:1347] Generated new cluster ID: c074c3315f3543b7862376f699dee565
I20260811 03:01:39.209497 16901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:39.233798 16901 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:39.234717 16901 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:39.248136 16901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e036e13f684264a9795615d5ca32de: Generated new TSK 0
I20260811 03:01:39.248791 16901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:39.253496 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 16723
W20260811 03:01:39.268935 16697 connection.cc:535] client connection to 127.0.0.1:42711 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42711): Connection reset by peer (error 104)
2026-08-11T03:01:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1604 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:39 dist-test-slave-gjw7 krb5kdc[16924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:39 dist-test-slave-gjw7 krb5kdc[16924](info): set up 2 sockets
Aug 11 03:01:39 dist-test-slave-gjw7 krb5kdc[16924](info): commencing operation
krb5kdc: starting...
W20260811 03:01:40.467782 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:40 dist-test-slave-gjw7 krb5kdc[16924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:40.583928 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:40.694046 16942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:40.694295 16942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:40.694351 16942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:40.698325 16942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:40.698378 16942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:40.698405 16942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:40.698426 16942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:40.698455 16942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:40.703495 16942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36383
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:40.704490 16942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:40.705510 16942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:40.713590 16950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:40.713626 16942 server_base.cc:1034] running on GCE node
W20260811 03:01:40.713582 16954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:40.713580 16958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:40.714314 16942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:40.715346 16942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:40.716504 16942 hybrid_clock.cc:648] HybridClock initialized: now 1786417300716488 us; error 30 us; skew 500 ppm
Aug 11 03:01:40 dist-test-slave-gjw7 krb5kdc[16924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417300, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:40.721927 16942 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:40.722754 16942 webserver.cc:460] Webserver started at http://127.0.0.1:37717/ using document root <none> and password file <none>
I20260811 03:01:40.723258 16942 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:40.723315 16942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:40.723480 16942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:40.725464 16942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "6626023b7c7d4da686d0aa823d061356"
format_stamp: "Formatted at 2026-08-11 03:01:40 on dist-test-slave-gjw7"
server_key: "7ab9c923c644a8c1a9cfb57fa7af3fea"
server_key_iv: "14eeb1ffb3cb809ddf8b1ad964e23890"
server_key_version: "encryptionkey@0"
I20260811 03:01:40.725993 16942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "6626023b7c7d4da686d0aa823d061356"
format_stamp: "Formatted at 2026-08-11 03:01:40 on dist-test-slave-gjw7"
server_key: "7ab9c923c644a8c1a9cfb57fa7af3fea"
server_key_iv: "14eeb1ffb3cb809ddf8b1ad964e23890"
server_key_version: "encryptionkey@0"
I20260811 03:01:40.729525 16942 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:01:40.731742 16970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:40.732997 16942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:40.733095 16942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "6626023b7c7d4da686d0aa823d061356"
format_stamp: "Formatted at 2026-08-11 03:01:40 on dist-test-slave-gjw7"
server_key: "7ab9c923c644a8c1a9cfb57fa7af3fea"
server_key_iv: "14eeb1ffb3cb809ddf8b1ad964e23890"
server_key_version: "encryptionkey@0"
I20260811 03:01:40.733178 16942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:40.737469 16942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:40.740049 16942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:40.740186 16942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:40.747002 16942 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36383
I20260811 03:01:40.747022 17076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36383 every 8 connection(s)
I20260811 03:01:40.747931 16942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:40.748941 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 16942
I20260811 03:01:40.749059 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:40.749361 9005 external_mini_cluster.cc:1428] Setting key 5093e309ec6e82eb83e59f558d8515c0
I20260811 03:01:40.750988 17078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:40.754356 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:40.750631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55583 (local address 127.0.0.1:36383)
0811 03:01:40.751030 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:40.751046 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:40.751084 (+ 38us) server_negotiation.cc:407] Connection header received
0811 03:01:40.751738 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:40.751754 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:40.752050 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:40.752324 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:40.753146 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:40.753151 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:40.753164 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:40.753175 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:40.753228 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:40.753245 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:40.753392 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:40.753484 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 03:01:40.753625 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260811 03:01:40.757138 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:40.757335 17078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356: Bootstrap starting.
I20260811 03:01:40.759979 17078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:40.760820 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:40.759390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57201 (local address 127.0.0.1:36383)
0811 03:01:40.759502 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:40.759506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:40.759649 (+ 143us) server_negotiation.cc:407] Connection header received
0811 03:01:40.759736 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:40.759739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:40.759771 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:40.759851 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:40.760494 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:40.760497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:40.760500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:40.760512 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:40.760540 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:40.760551 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:40.760638 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:40.760674 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 03:01:40.760710 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:01:40.760830 17078 log.cc:824] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:40.762681 17078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356: No bootstrap required, opened a new log
I20260811 03:01:40.765986 17078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6626023b7c7d4da686d0aa823d061356" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260811 03:01:40.766199 17078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:40.766239 17078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6626023b7c7d4da686d0aa823d061356, State: Initialized, Role: FOLLOWER
I20260811 03:01:40.766585 17078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6626023b7c7d4da686d0aa823d061356" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260811 03:01:40.766708 17078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:40.766752 17078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:40.766820 17078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:40.767688 17078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6626023b7c7d4da686d0aa823d061356" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260811 03:01:40.767985 17078 leader_election.cc:302] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6626023b7c7d4da686d0aa823d061356; no voters:
I20260811 03:01:40.768216 17078 leader_election.cc:288] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:40.768314 17103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:40.768519 17103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [term 1 LEADER]: Becoming Leader. State: Replica: 6626023b7c7d4da686d0aa823d061356, State: Running, Role: LEADER
I20260811 03:01:40.768833 17103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6626023b7c7d4da686d0aa823d061356" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } }
I20260811 03:01:40.769340 17078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:40.770406 17115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6626023b7c7d4da686d0aa823d061356. Latest consensus state: current_term: 1 leader_uuid: "6626023b7c7d4da686d0aa823d061356" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6626023b7c7d4da686d0aa823d061356" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260811 03:01:40.770533 17115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:40.770905 17126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:40.770869 17114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6626023b7c7d4da686d0aa823d061356" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6626023b7c7d4da686d0aa823d061356" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36383 } } }
I20260811 03:01:40.771014 17114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:40.773802 17126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:40.779237 17126 catalog_manager.cc:1347] Generated new cluster ID: 9ad7f883ce71401298440ed064657a49
I20260811 03:01:40.779301 17126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:40.791586 17126 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:40.791867 17126 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:40.799031 17126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6626023b7c7d4da686d0aa823d061356: Generated new TSK 0
I20260811 03:01:40.799644 17126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:40.809875 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 16942
2026-08-11T03:01:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:40 dist-test-slave-gjw7 krb5kdc[17155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:40 dist-test-slave-gjw7 krb5kdc[17155](info): set up 2 sockets
Aug 11 03:01:40 dist-test-slave-gjw7 krb5kdc[17155](info): commencing operation
krb5kdc: starting...
W20260811 03:01:42.037181 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:42 dist-test-slave-gjw7 krb5kdc[17155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:42.155512 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:42.263792 17179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:42.264058 17179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:42.264098 17179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:42.267974 17179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:42.268026 17179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:42.268044 17179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:42.268062 17179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:42.268078 17179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:42.273150 17179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:42.274178 17179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:42.275127 17179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:42.283242 17190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:42.283327 17194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:42.283291 17192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:42.283398 17179 server_base.cc:1034] running on GCE node
I20260811 03:01:42.284032 17179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:42.285085 17179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:42.286235 17179 hybrid_clock.cc:648] HybridClock initialized: now 1786417302286216 us; error 31 us; skew 500 ppm
Aug 11 03:01:42 dist-test-slave-gjw7 krb5kdc[17155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417302, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:42.288805 17179 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:42.289660 17179 webserver.cc:460] Webserver started at http://127.0.0.1:39515/ using document root <none> and password file <none>
I20260811 03:01:42.290182 17179 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:42.290241 17179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:42.290439 17179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:42.292387 17179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "f7b36a8b64f043389e59a92dc256ab0b"
format_stamp: "Formatted at 2026-08-11 03:01:42 on dist-test-slave-gjw7"
server_key: "0f9123d571cb2434da30d63207b6dda8"
server_key_iv: "008daceef37339395a26815a2f871ba0"
server_key_version: "encryptionkey@0"
I20260811 03:01:42.292914 17179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "f7b36a8b64f043389e59a92dc256ab0b"
format_stamp: "Formatted at 2026-08-11 03:01:42 on dist-test-slave-gjw7"
server_key: "0f9123d571cb2434da30d63207b6dda8"
server_key_iv: "008daceef37339395a26815a2f871ba0"
server_key_version: "encryptionkey@0"
I20260811 03:01:42.296397 17179 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:01:42.298704 17205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:42.299822 17179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:42.299964 17179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "f7b36a8b64f043389e59a92dc256ab0b"
format_stamp: "Formatted at 2026-08-11 03:01:42 on dist-test-slave-gjw7"
server_key: "0f9123d571cb2434da30d63207b6dda8"
server_key_iv: "008daceef37339395a26815a2f871ba0"
server_key_version: "encryptionkey@0"
I20260811 03:01:42.300056 17179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:42.315546 17179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:42.318161 17179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:42.318322 17179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:42.324785 17179 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42135
I20260811 03:01:42.324810 17290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42135 every 8 connection(s)
I20260811 03:01:42.325685 17179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:42.328289 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:42.330866 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17179
I20260811 03:01:42.330986 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:42.331230 9005 external_mini_cluster.cc:1428] Setting key 25bb09ff5be10e1ef01afc182d9cf782
I20260811 03:01:42.334170 17293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b: Bootstrap starting.
I20260811 03:01:42.336750 17293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:42.337515 17293 log.cc:824] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:42.339346 17293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b: No bootstrap required, opened a new log
Aug 11 03:01:42 dist-test-slave-gjw7 krb5kdc[17155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:01:42.342804 17293 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b36a8b64f043389e59a92dc256ab0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260811 03:01:42.343071 17293 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:42.343129 17293 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b36a8b64f043389e59a92dc256ab0b, State: Initialized, Role: FOLLOWER
I20260811 03:01:42.343552 17293 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b36a8b64f043389e59a92dc256ab0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260811 03:01:42.343709 17293 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:42.343780 17293 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:42.343880 17293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:42.344916 17293 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b36a8b64f043389e59a92dc256ab0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260811 03:01:42.345260 17293 leader_election.cc:302] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7b36a8b64f043389e59a92dc256ab0b; no voters:
I20260811 03:01:42.345602 17293 leader_election.cc:288] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:42.345705 17305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:42.345917 17305 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [term 1 LEADER]: Becoming Leader. State: Replica: f7b36a8b64f043389e59a92dc256ab0b, State: Running, Role: LEADER
I20260811 03:01:42.346201 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:42.332558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50339 (local address 127.0.0.1:42135)
0811 03:01:42.332981 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:42.333001 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:42.333044 (+ 43us) server_negotiation.cc:407] Connection header received
0811 03:01:42.333847 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:42.333882 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:42.334300 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:42.334649 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:42.335286 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:42.336750 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:42.337646 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:42.338187 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:42.338342 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:42.340581 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:42.340587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:42.340604 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:42.340634 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:42.343118 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:42.343565 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:42.343572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:42.343578 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:42.343719 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:42.344016 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:42.344019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:42.344021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:42.344503 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:42.344711 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:42.344888 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 03:01:42.345185 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
I20260811 03:01:42.346269 17305 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b36a8b64f043389e59a92dc256ab0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260811 03:01:42.346817 17293 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:42.347947 17319 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7b36a8b64f043389e59a92dc256ab0b. Latest consensus state: current_term: 1 leader_uuid: "f7b36a8b64f043389e59a92dc256ab0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b36a8b64f043389e59a92dc256ab0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } } }
I20260811 03:01:42.348067 17319 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:42.348389 17318 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7b36a8b64f043389e59a92dc256ab0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b36a8b64f043389e59a92dc256ab0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } } }
I20260811 03:01:42.348466 17318 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:42.348824 17328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:42.349570 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:42.351807 17328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:42.357584 17328 catalog_manager.cc:1347] Generated new cluster ID: 3cc9dd2cf218417da76764a12aa66327
I20260811 03:01:42.357674 17328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:42.361265 17299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:42.351639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54093 (local address 127.0.0.1:42135)
0811 03:01:42.351738 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:42.351742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:42.353578 (+ 1836us) server_negotiation.cc:407] Connection header received
0811 03:01:42.353713 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:42.353716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:42.353764 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:42.353847 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:42.354628 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:42.355300 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:42.356083 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:42.356413 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:42.356515 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:42.358129 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:42.358133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:42.358136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:42.358162 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:42.359714 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:42.360168 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:42.360172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:42.360173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:42.360245 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:42.360499 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:42.360502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:42.360504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:42.360677 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:42.360777 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:42.361022 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 03:01:42.361123 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 03:01:42.371299 17328 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:42.372519 17328 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:42.381107 17328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7b36a8b64f043389e59a92dc256ab0b: Generated new TSK 0
I20260811 03:01:42.381804 17328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:42.391583 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17179
2026-08-11T03:01:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:42 dist-test-slave-gjw7 krb5kdc[17364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:42 dist-test-slave-gjw7 krb5kdc[17364](info): set up 2 sockets
Aug 11 03:01:42 dist-test-slave-gjw7 krb5kdc[17364](info): commencing operation
krb5kdc: starting...
W20260811 03:01:43.606498 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:43 dist-test-slave-gjw7 krb5kdc[17364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:43.723239 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41183
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:43.830401 17398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:43.830644 17398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:43.830699 17398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:43.834630 17398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:43.834686 17398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:43.834712 17398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:43.834735 17398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:43.834764 17398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:43.840683 17398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:43.841694 17398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:43.842653 17398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:43.850564 17406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:43.850540 17411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:43.850533 17407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:43.850814 17398 server_base.cc:1034] running on GCE node
I20260811 03:01:43.851327 17398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:43.852299 17398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:43.853443 17398 hybrid_clock.cc:648] HybridClock initialized: now 1786417303853426 us; error 29 us; skew 500 ppm
Aug 11 03:01:43 dist-test-slave-gjw7 krb5kdc[17364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417303, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:43.855885 17398 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:43.856715 17398 webserver.cc:460] Webserver started at http://127.0.0.1:43739/ using document root <none> and password file <none>
I20260811 03:01:43.857195 17398 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:43.857249 17398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:43.857466 17398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:43.859357 17398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "9149f7ba7cf0488d8c7921df00bab00c"
format_stamp: "Formatted at 2026-08-11 03:01:43 on dist-test-slave-gjw7"
server_key: "264996cfeeaa5455ccfff63f4c28136b"
server_key_iv: "ef000480e51b08f6b4d1a3e2fe90d3c8"
server_key_version: "encryptionkey@0"
I20260811 03:01:43.859853 17398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "9149f7ba7cf0488d8c7921df00bab00c"
format_stamp: "Formatted at 2026-08-11 03:01:43 on dist-test-slave-gjw7"
server_key: "264996cfeeaa5455ccfff63f4c28136b"
server_key_iv: "ef000480e51b08f6b4d1a3e2fe90d3c8"
server_key_version: "encryptionkey@0"
I20260811 03:01:43.863242 17398 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 03:01:43.865485 17426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:43.866605 17398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:43.866705 17398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "9149f7ba7cf0488d8c7921df00bab00c"
format_stamp: "Formatted at 2026-08-11 03:01:43 on dist-test-slave-gjw7"
server_key: "264996cfeeaa5455ccfff63f4c28136b"
server_key_iv: "ef000480e51b08f6b4d1a3e2fe90d3c8"
server_key_version: "encryptionkey@0"
I20260811 03:01:43.866801 17398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:43.898799 17398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:43.901564 17398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:43.901731 17398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:43.908452 17398 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41183
I20260811 03:01:43.908466 17536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41183 every 8 connection(s)
I20260811 03:01:43.909422 17398 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:43.912101 17537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:43.917498 17537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c: Bootstrap starting.
I20260811 03:01:43.918479 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17398
I20260811 03:01:43.918628 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:43.918912 9005 external_mini_cluster.cc:1428] Setting key 0c63bce5c4807e7fe6d5dc1566023941
I20260811 03:01:43.920261 17537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:43.920954 17537 log.cc:824] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:43.923009 17537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c: No bootstrap required, opened a new log
I20260811 03:01:43.926959 17537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9149f7ba7cf0488d8c7921df00bab00c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41183 } }
I20260811 03:01:43.927166 17537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:43.927201 17537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9149f7ba7cf0488d8c7921df00bab00c, State: Initialized, Role: FOLLOWER
I20260811 03:01:43.927379 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:43.919966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40825 (local address 127.0.0.1:41183)
0811 03:01:43.920341 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:43.920361 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:43.920406 (+ 45us) server_negotiation.cc:407] Connection header received
0811 03:01:43.921012 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:43.921033 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:43.921478 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:43.921816 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:43.922144 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:43.923130 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:43.924294 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:43.924753 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:43.924863 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:43.925737 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:43.925742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:43.925755 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:43.926123 (+ 368us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:43.926174 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:43.926196 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:43.926270 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:43.926428 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 03:01:43.926622 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":91,"threads_started":1}
I20260811 03:01:43.927649 17537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9149f7ba7cf0488d8c7921df00bab00c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41183 } }
I20260811 03:01:43.927786 17537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:43.927860 17537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:43.927958 17537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:43.928921 17537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9149f7ba7cf0488d8c7921df00bab00c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41183 } }
I20260811 03:01:43.929257 17537 leader_election.cc:302] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9149f7ba7cf0488d8c7921df00bab00c; no voters:
I20260811 03:01:43.929600 17537 leader_election.cc:288] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:43.929682 17549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:43.929899 17549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [term 1 LEADER]: Becoming Leader. State: Replica: 9149f7ba7cf0488d8c7921df00bab00c, State: Running, Role: LEADER
I20260811 03:01:43.930274 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:43.930258 17549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9149f7ba7cf0488d8c7921df00bab00c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41183 } }
I20260811 03:01:43.930768 17537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:43.932621 17552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9149f7ba7cf0488d8c7921df00bab00c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9149f7ba7cf0488d8c7921df00bab00c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41183 } } }
I20260811 03:01:43.933228 17552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:43.932701 17550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9149f7ba7cf0488d8c7921df00bab00c. Latest consensus state: current_term: 1 leader_uuid: "9149f7ba7cf0488d8c7921df00bab00c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9149f7ba7cf0488d8c7921df00bab00c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41183 } } }
I20260811 03:01:43.933488 17550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c [sys.catalog]: This master's current role is: LEADER
W20260811 03:01:43.935773 17589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:01:43.935850 17589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:01:43.936123 17593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:43.936957 17593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:43.937198 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:43.932990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56969 (local address 127.0.0.1:41183)
0811 03:01:43.933101 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:43.933105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:43.933208 (+ 103us) server_negotiation.cc:407] Connection header received
0811 03:01:43.933348 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:43.933351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:43.933395 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:43.933466 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:43.933811 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:43.934523 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:43.935578 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:43.935889 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:43.936006 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:43.936782 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:43.936786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:43.936789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:43.936802 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:43.936829 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:43.936843 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:43.936917 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:43.937030 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 03:01:43.937081 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:01:43.942237 17593 catalog_manager.cc:1347] Generated new cluster ID: db750598ddb742c2a76f8d82834370f9
I20260811 03:01:43.942286 17593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:43.953437 17593 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:43.954267 17593 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:43.959348 17593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9149f7ba7cf0488d8c7921df00bab00c: Generated new TSK 0
I20260811 03:01:43.959928 17593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:43.969725 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17398
2026-08-11T03:01:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:44 dist-test-slave-gjw7 krb5kdc[17605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:44 dist-test-slave-gjw7 krb5kdc[17605](info): set up 2 sockets
Aug 11 03:01:44 dist-test-slave-gjw7 krb5kdc[17605](info): commencing operation
krb5kdc: starting...
W20260811 03:01:45.170352 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:45 dist-test-slave-gjw7 krb5kdc[17605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:45.284785 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33407
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:45.391866 17628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:45.392123 17628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:45.392201 17628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:45.396112 17628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:45.396184 17628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:45.396216 17628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:45.396253 17628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:45.396288 17628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:45.401463 17628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:45.402455 17628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:45.403465 17628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:45.411476 17643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:45.411541 17644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:45.411471 17646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:45.411831 17628 server_base.cc:1034] running on GCE node
I20260811 03:01:45.412340 17628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:45.413362 17628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:45.414511 17628 hybrid_clock.cc:648] HybridClock initialized: now 1786417305414495 us; error 35 us; skew 500 ppm
Aug 11 03:01:45 dist-test-slave-gjw7 krb5kdc[17605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417305, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:45.417089 17628 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:45.417929 17628 webserver.cc:460] Webserver started at http://127.0.0.1:44243/ using document root <none> and password file <none>
I20260811 03:01:45.418430 17628 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:45.418495 17628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:45.418695 17628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:45.420635 17628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "5b6c6edcd5504ca7b32300a17a94dc4f"
format_stamp: "Formatted at 2026-08-11 03:01:45 on dist-test-slave-gjw7"
server_key: "ae52cb17f2d2993ff3f1ec12dc1f0c8f"
server_key_iv: "133aa23f56ca99816ef8240383550aa4"
server_key_version: "encryptionkey@0"
I20260811 03:01:45.421157 17628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "5b6c6edcd5504ca7b32300a17a94dc4f"
format_stamp: "Formatted at 2026-08-11 03:01:45 on dist-test-slave-gjw7"
server_key: "ae52cb17f2d2993ff3f1ec12dc1f0c8f"
server_key_iv: "133aa23f56ca99816ef8240383550aa4"
server_key_version: "encryptionkey@0"
I20260811 03:01:45.424540 17628 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 03:01:45.426791 17659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:45.427891 17628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 03:01:45.427999 17628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "5b6c6edcd5504ca7b32300a17a94dc4f"
format_stamp: "Formatted at 2026-08-11 03:01:45 on dist-test-slave-gjw7"
server_key: "ae52cb17f2d2993ff3f1ec12dc1f0c8f"
server_key_iv: "133aa23f56ca99816ef8240383550aa4"
server_key_version: "encryptionkey@0"
I20260811 03:01:45.428085 17628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:45.432363 17628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:45.434942 17628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:45.435091 17628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:45.441376 17628 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33407
I20260811 03:01:45.441418 17769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33407 every 8 connection(s)
I20260811 03:01:45.442207 17628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:45.444797 17772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:45.449514 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17628
I20260811 03:01:45.449642 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:45.449873 9005 external_mini_cluster.cc:1428] Setting key 8478e13dd8f8b315d9dbc638f63526a5
I20260811 03:01:45.450337 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f: Bootstrap starting.
I20260811 03:01:45.452951 17772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:45.453897 17772 log.cc:824] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:45.455207 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:45.451157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59287 (local address 127.0.0.1:33407)
0811 03:01:45.451610 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:45.451630 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:45.451674 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:01:45.452524 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:45.452547 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:45.452895 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:45.453285 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:45.453940 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:45.453946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:45.453961 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:45.453974 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:45.454037 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:45.454067 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:45.454190 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:45.454272 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 03:01:45.454409 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":114,"threads_started":1}
I20260811 03:01:45.456001 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f: No bootstrap required, opened a new log
I20260811 03:01:45.457964 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:45.460410 17772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260811 03:01:45.460692 17772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:45.460763 17772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b6c6edcd5504ca7b32300a17a94dc4f, State: Initialized, Role: FOLLOWER
I20260811 03:01:45.461182 17772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260811 03:01:45.461342 17772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:45.461417 17772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:45.461524 17772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:45.461750 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:45.460283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52991 (local address 127.0.0.1:33407)
0811 03:01:45.460396 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:45.460401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:45.460487 (+ 86us) server_negotiation.cc:407] Connection header received
0811 03:01:45.460577 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:45.460580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:45.460615 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:45.460705 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:45.461355 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:45.461360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:45.461362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:45.461376 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:45.461406 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:45.461422 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:45.461528 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:45.461598 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 03:01:45.461637 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 03:01:45.462567 17772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260811 03:01:45.462880 17772 leader_election.cc:302] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b6c6edcd5504ca7b32300a17a94dc4f; no voters:
I20260811 03:01:45.463127 17772 leader_election.cc:288] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:45.463231 17793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:45.463419 17793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [term 1 LEADER]: Becoming Leader. State: Replica: 5b6c6edcd5504ca7b32300a17a94dc4f, State: Running, Role: LEADER
I20260811 03:01:45.463752 17793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } }
I20260811 03:01:45.464214 17772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:45.465216 17797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b6c6edcd5504ca7b32300a17a94dc4f. Latest consensus state: current_term: 1 leader_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } } }
I20260811 03:01:45.465374 17797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:45.465615 17794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b6c6edcd5504ca7b32300a17a94dc4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33407 } } }
I20260811 03:01:45.465742 17794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:45.466552 17817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:45.468961 17817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:45.473616 17817 catalog_manager.cc:1347] Generated new cluster ID: debcfa096330449182fb8aa8290c4ffb
I20260811 03:01:45.473683 17817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:45.482372 17817 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:45.482627 17817 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:45.486994 17817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b6c6edcd5504ca7b32300a17a94dc4f: Generated new TSK 0
I20260811 03:01:45.487521 17817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:45.498580 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17628
2026-08-11T03:01:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:45 dist-test-slave-gjw7 krb5kdc[17837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:45 dist-test-slave-gjw7 krb5kdc[17837](info): set up 2 sockets
Aug 11 03:01:45 dist-test-slave-gjw7 krb5kdc[17837](info): commencing operation
krb5kdc: starting...
W20260811 03:01:46.712831 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:46 dist-test-slave-gjw7 krb5kdc[17837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:46.829708 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33397
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:46.940188 17857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:46.940420 17857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:46.940462 17857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:46.944295 17857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:46.944343 17857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:46.944361 17857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:46.944398 17857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:46.944424 17857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:46.949471 17857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:46.950433 17857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:46.951435 17857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:46.959563 17874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:46.959573 17875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:46.959574 17877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:46.959849 17857 server_base.cc:1034] running on GCE node
I20260811 03:01:46.960404 17857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:46.961444 17857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:46.962594 17857 hybrid_clock.cc:648] HybridClock initialized: now 1786417306962573 us; error 34 us; skew 500 ppm
Aug 11 03:01:46 dist-test-slave-gjw7 krb5kdc[17837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417306, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:46.968199 17857 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:46.969058 17857 webserver.cc:460] Webserver started at http://127.0.0.1:39095/ using document root <none> and password file <none>
I20260811 03:01:46.969612 17857 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:46.969679 17857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:46.969890 17857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:46.971846 17857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "b95a925ee1c8424388e00cd276099492"
format_stamp: "Formatted at 2026-08-11 03:01:46 on dist-test-slave-gjw7"
server_key: "9390defe34c7dddc86d3ae7fa14810ca"
server_key_iv: "7a1ba044a34c0177c0a3d8b444d7dd48"
server_key_version: "encryptionkey@0"
I20260811 03:01:46.972370 17857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "b95a925ee1c8424388e00cd276099492"
format_stamp: "Formatted at 2026-08-11 03:01:46 on dist-test-slave-gjw7"
server_key: "9390defe34c7dddc86d3ae7fa14810ca"
server_key_iv: "7a1ba044a34c0177c0a3d8b444d7dd48"
server_key_version: "encryptionkey@0"
I20260811 03:01:46.975826 17857 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:01:46.978077 17895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:46.979187 17857 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 03:01:46.979295 17857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "b95a925ee1c8424388e00cd276099492"
format_stamp: "Formatted at 2026-08-11 03:01:46 on dist-test-slave-gjw7"
server_key: "9390defe34c7dddc86d3ae7fa14810ca"
server_key_iv: "7a1ba044a34c0177c0a3d8b444d7dd48"
server_key_version: "encryptionkey@0"
I20260811 03:01:46.979383 17857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:46.996160 17857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:46.999027 17857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:46.999187 17857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:47.005853 17857 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33397
I20260811 03:01:47.005932 17994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33397 every 8 connection(s)
I20260811 03:01:47.006760 17857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:47.009392 17999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:47.014658 17999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492: Bootstrap starting.
I20260811 03:01:47.015615 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 17857
I20260811 03:01:47.015729 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:47.016000 9005 external_mini_cluster.cc:1428] Setting key b9baf4d41eedf7f6acf984558b623ae0
I20260811 03:01:47.017104 17999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:47.017908 17999 log.cc:824] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:47.019733 17999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492: No bootstrap required, opened a new log
I20260811 03:01:47.023043 17999 raft_consensus.cc:348] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95a925ee1c8424388e00cd276099492" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33397 } }
I20260811 03:01:47.023258 17999 raft_consensus.cc:374] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:47.023315 17999 raft_consensus.cc:729] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b95a925ee1c8424388e00cd276099492, State: Initialized, Role: FOLLOWER
I20260811 03:01:47.023747 17999 consensus_queue.cc:260] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95a925ee1c8424388e00cd276099492" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33397 } }
I20260811 03:01:47.023893 17999 raft_consensus.cc:388] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:47.023960 17999 raft_consensus.cc:482] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:47.024063 17999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 03:01:47 dist-test-slave-gjw7 krb5kdc[17837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:01:47.025008 17999 raft_consensus.cc:504] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95a925ee1c8424388e00cd276099492" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33397 } }
I20260811 03:01:47.025359 17999 leader_election.cc:302] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b95a925ee1c8424388e00cd276099492; no voters:
I20260811 03:01:47.025641 17999 leader_election.cc:288] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:47.025791 18009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:47.026240 18009 raft_consensus.cc:686] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [term 1 LEADER]: Becoming Leader. State: Replica: b95a925ee1c8424388e00cd276099492, State: Running, Role: LEADER
I20260811 03:01:47.026618 18009 consensus_queue.cc:237] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95a925ee1c8424388e00cd276099492" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33397 } }
I20260811 03:01:47.026830 17999 sys_catalog.cc:564] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:47.028278 18011 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b95a925ee1c8424388e00cd276099492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95a925ee1c8424388e00cd276099492" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33397 } } }
I20260811 03:01:47.028400 18011 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:47.028784 18012 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b95a925ee1c8424388e00cd276099492. Latest consensus state: current_term: 1 leader_uuid: "b95a925ee1c8424388e00cd276099492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95a925ee1c8424388e00cd276099492" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33397 } } }
I20260811 03:01:47.028996 18012 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:47.029645 18029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:47.030889 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:47.017105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:36791 (local address 127.0.0.1:33397)
0811 03:01:47.017522 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:47.017536 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:47.017569 (+ 33us) server_negotiation.cc:407] Connection header received
0811 03:01:47.018243 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:47.018270 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:47.018717 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:47.019031 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:47.019831 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:47.020800 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:47.021813 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:47.022289 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:47.022434 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:47.025072 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:47.025078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:47.025095 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:47.025128 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:47.027145 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:47.027793 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:47.027800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:47.027806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:47.027978 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:47.028394 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:47.028398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:47.028399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:47.028954 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:47.029178 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:47.029975 (+ 797us) server_negotiation.cc:299] Negotiation successful
0811 03:01:47.030153 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":139,"threads_started":1}
I20260811 03:01:47.032612 18029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:47.035707 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:47.039126 18029 catalog_manager.cc:1347] Generated new cluster ID: ef0649e7997344bd95c57dd97ce33727
I20260811 03:01:47.039186 18029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:47.045907 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:47.037901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:60661 (local address 127.0.0.1:33397)
0811 03:01:47.038011 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:47.038015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:47.038341 (+ 326us) server_negotiation.cc:407] Connection header received
0811 03:01:47.038473 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:47.038476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:47.038534 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:47.038618 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:47.039448 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:47.039997 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:47.040826 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:47.041066 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:47.041164 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:47.042629 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:47.042633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:47.042636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:47.042672 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:47.044051 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:47.044555 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:47.044558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:47.044559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:47.044641 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:47.044986 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:47.044990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:47.044992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:47.045169 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:47.045351 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:47.045632 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 03:01:47.045737 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 03:01:47.075748 18029 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:47.076869 18029 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:47.080968 18029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b95a925ee1c8424388e00cd276099492: Generated new TSK 0
I20260811 03:01:47.081643 18029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:47.087499 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 17857
W20260811 03:01:47.107503 17829 connection.cc:535] client connection to 127.0.0.1:33397 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33397): Connection reset by peer (error 104)
2026-08-11T03:01:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1593 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:47 dist-test-slave-gjw7 krb5kdc[18049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:47 dist-test-slave-gjw7 krb5kdc[18049](info): set up 2 sockets
Aug 11 03:01:47 dist-test-slave-gjw7 krb5kdc[18049](info): commencing operation
krb5kdc: starting...
W20260811 03:01:48.309527 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:48 dist-test-slave-gjw7 krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:48.425726 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:48.533604 18081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:48.533830 18081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:48.533880 18081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:48.537781 18081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:48.537840 18081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:48.537860 18081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:48.537899 18081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:48.537925 18081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:48.543067 18081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:48.544101 18081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:48.545071 18081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:48.553097 18105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:48.553117 18100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:48.553229 18081 server_base.cc:1034] running on GCE node
W20260811 03:01:48.553097 18098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:48.553956 18081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:48.554955 18081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:48.556106 18081 hybrid_clock.cc:648] HybridClock initialized: now 1786417308556092 us; error 24 us; skew 500 ppm
Aug 11 03:01:48 dist-test-slave-gjw7 krb5kdc[18049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417308, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:48.558737 18081 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:48.559587 18081 webserver.cc:460] Webserver started at http://127.0.0.1:39565/ using document root <none> and password file <none>
I20260811 03:01:48.560149 18081 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:48.560218 18081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:48.560425 18081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:48.562479 18081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "a4ff059d07c24bafa7eb0862a4f9ced8"
format_stamp: "Formatted at 2026-08-11 03:01:48 on dist-test-slave-gjw7"
server_key: "69c704514bdcbfee93aeb8e1f998801e"
server_key_iv: "bc3f61405fe1235dd6a30820c53d350f"
server_key_version: "encryptionkey@0"
I20260811 03:01:48.563032 18081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "a4ff059d07c24bafa7eb0862a4f9ced8"
format_stamp: "Formatted at 2026-08-11 03:01:48 on dist-test-slave-gjw7"
server_key: "69c704514bdcbfee93aeb8e1f998801e"
server_key_iv: "bc3f61405fe1235dd6a30820c53d350f"
server_key_version: "encryptionkey@0"
I20260811 03:01:48.566640 18081 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:01:48.568912 18114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:48.570063 18081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:48.570176 18081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "a4ff059d07c24bafa7eb0862a4f9ced8"
format_stamp: "Formatted at 2026-08-11 03:01:48 on dist-test-slave-gjw7"
server_key: "69c704514bdcbfee93aeb8e1f998801e"
server_key_iv: "bc3f61405fe1235dd6a30820c53d350f"
server_key_version: "encryptionkey@0"
I20260811 03:01:48.570276 18081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:48.587178 18081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:48.590258 18081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:48.590435 18081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:48.597038 18081 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39153
I20260811 03:01:48.597080 18218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39153 every 8 connection(s)
I20260811 03:01:48.597982 18081 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:48.600697 18220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:48.600744 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18081
I20260811 03:01:48.600878 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:48.601186 9005 external_mini_cluster.cc:1428] Setting key 43ed2e7b61f695c4b98492cbd3b2aa34
I20260811 03:01:48.607204 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8: Bootstrap starting.
I20260811 03:01:48.609604 18220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:48.609946 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:48.602520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:45427 (local address 127.0.0.1:39153)
0811 03:01:48.602912 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:48.602927 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:48.602963 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:01:48.603783 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:48.603802 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:48.604138 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:48.604415 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:48.604720 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:48.605717 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:48.606876 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:48.607318 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:48.607461 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:48.608273 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:48.608278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:48.608290 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:48.608661 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:48.608715 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:48.608739 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:48.608835 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:48.609020 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 03:01:48.609155 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":109,"threads_started":1}
I20260811 03:01:48.610328 18220 log.cc:824] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:48.612150 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8: No bootstrap required, opened a new log
I20260811 03:01:48.612582 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:48.616273 18220 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39153 } }
I20260811 03:01:48.616604 18220 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:48.616671 18220 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ff059d07c24bafa7eb0862a4f9ced8, State: Initialized, Role: FOLLOWER
I20260811 03:01:48.617138 18220 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39153 } }
I20260811 03:01:48.617347 18220 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:48.617451 18220 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:48.617556 18220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:48.618637 18220 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39153 } }
I20260811 03:01:48.618791 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:48.614724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:41547 (local address 127.0.0.1:39153)
0811 03:01:48.614838 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:48.614843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:48.615096 (+ 253us) server_negotiation.cc:407] Connection header received
0811 03:01:48.615203 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:48.615205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:48.615243 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:48.615304 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:48.615549 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:48.616034 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:48.617087 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:48.617498 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:48.617597 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:48.618384 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:48.618387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:48.618399 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:48.618411 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:48.618433 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:48.618442 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:48.618506 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:48.618643 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 03:01:48.618669 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 03:01:48.619014 18220 leader_election.cc:302] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4ff059d07c24bafa7eb0862a4f9ced8; no voters:
I20260811 03:01:48.619321 18220 leader_election.cc:288] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:48.619402 18237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:48.619601 18237 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [term 1 LEADER]: Becoming Leader. State: Replica: a4ff059d07c24bafa7eb0862a4f9ced8, State: Running, Role: LEADER
I20260811 03:01:48.620042 18237 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39153 } }
I20260811 03:01:48.620589 18220 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:48.621249 18237 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4ff059d07c24bafa7eb0862a4f9ced8. Latest consensus state: current_term: 1 leader_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39153 } } }
I20260811 03:01:48.621393 18237 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:48.622933 18270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:48.623101 18239 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ff059d07c24bafa7eb0862a4f9ced8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39153 } } }
I20260811 03:01:48.624125 18239 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:48.626570 18270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:48.632054 18270 catalog_manager.cc:1347] Generated new cluster ID: cdecf7bb83e4422da5365c2789acb31b
I20260811 03:01:48.632105 18270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:48.649786 18270 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:48.650707 18270 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:48.669659 18270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4ff059d07c24bafa7eb0862a4f9ced8: Generated new TSK 0
I20260811 03:01:48.670339 18270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:48.680735 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18081
W20260811 03:01:48.695621 18047 connection.cc:535] client connection to 127.0.0.1:39153 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39153): Connection reset by peer (error 104)
2026-08-11T03:01:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1588 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:48 dist-test-slave-gjw7 krb5kdc[18284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:48 dist-test-slave-gjw7 krb5kdc[18284](info): set up 2 sockets
Aug 11 03:01:48 dist-test-slave-gjw7 krb5kdc[18284](info): commencing operation
krb5kdc: starting...
W20260811 03:01:49.888814 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:49 dist-test-slave-gjw7 krb5kdc[18284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:50.006112 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34367
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:50.116147 18312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:50.116376 18312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:50.116415 18312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:50.120260 18312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:50.120309 18312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:50.120327 18312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:50.120347 18312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:50.120361 18312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:50.125519 18312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:50.126413 18312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:50.127337 18312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:50.135224 18328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:50.135250 18324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:50.135245 18317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:50.135393 18312 server_base.cc:1034] running on GCE node
I20260811 03:01:50.136075 18312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:50.137118 18312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:50.138273 18312 hybrid_clock.cc:648] HybridClock initialized: now 1786417310138243 us; error 42 us; skew 500 ppm
Aug 11 03:01:50 dist-test-slave-gjw7 krb5kdc[18284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417310, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:50.140763 18312 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:50.141640 18312 webserver.cc:460] Webserver started at http://127.0.0.1:43737/ using document root <none> and password file <none>
I20260811 03:01:50.142167 18312 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:50.142232 18312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:50.142418 18312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:50.144335 18312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "fcf7d994a4824a538bca4fe6e20717b4"
format_stamp: "Formatted at 2026-08-11 03:01:50 on dist-test-slave-gjw7"
server_key: "b3b696766172007a9cbb648bae8a8202"
server_key_iv: "d236dc0d59e61176377b7303a3e87491"
server_key_version: "encryptionkey@0"
I20260811 03:01:50.144850 18312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "fcf7d994a4824a538bca4fe6e20717b4"
format_stamp: "Formatted at 2026-08-11 03:01:50 on dist-test-slave-gjw7"
server_key: "b3b696766172007a9cbb648bae8a8202"
server_key_iv: "d236dc0d59e61176377b7303a3e87491"
server_key_version: "encryptionkey@0"
I20260811 03:01:50.148327 18312 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:01:50.150547 18342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:50.151777 18312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:50.151892 18312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "fcf7d994a4824a538bca4fe6e20717b4"
format_stamp: "Formatted at 2026-08-11 03:01:50 on dist-test-slave-gjw7"
server_key: "b3b696766172007a9cbb648bae8a8202"
server_key_iv: "d236dc0d59e61176377b7303a3e87491"
server_key_version: "encryptionkey@0"
I20260811 03:01:50.151989 18312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:50.156458 18312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:50.159173 18312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:50.159335 18312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:50.166189 18312 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34367
I20260811 03:01:50.166229 18451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34367 every 8 connection(s)
I20260811 03:01:50.167093 18312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:50.169880 18455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:50.171764 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18312
I20260811 03:01:50.171875 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:50.172124 9005 external_mini_cluster.cc:1428] Setting key 999cbc5c4b582a50b6914ea184a0a828
I20260811 03:01:50.176121 18455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4: Bootstrap starting.
I20260811 03:01:50.177891 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:50.173557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:57853 (local address 127.0.0.1:34367)
0811 03:01:50.173988 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:50.174007 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:50.174049 (+ 42us) server_negotiation.cc:407] Connection header received
0811 03:01:50.175002 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:50.175026 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:50.175389 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:50.175747 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:50.176429 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:50.176435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:50.176449 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:50.176462 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:50.176525 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:50.176551 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:50.176684 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:50.176815 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 03:01:50.176972 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":119,"threads_started":1}
I20260811 03:01:50.178848 18455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:50.179601 18455 log.cc:824] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:50.180697 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:50.181874 18455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4: No bootstrap required, opened a new log
I20260811 03:01:50.184242 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:50.182659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:52347 (local address 127.0.0.1:34367)
0811 03:01:50.182748 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:50.182752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:50.182987 (+ 235us) server_negotiation.cc:407] Connection header received
0811 03:01:50.183113 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:50.183116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:50.183147 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:50.183206 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:50.183896 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:50.183899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:50.183902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:50.183913 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:50.183939 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:50.183950 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:50.184017 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:50.184099 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 03:01:50.184135 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 03:01:50.186015 18455 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf7d994a4824a538bca4fe6e20717b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34367 } }
I20260811 03:01:50.186254 18455 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:50.186328 18455 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf7d994a4824a538bca4fe6e20717b4, State: Initialized, Role: FOLLOWER
I20260811 03:01:50.186744 18455 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf7d994a4824a538bca4fe6e20717b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34367 } }
I20260811 03:01:50.186878 18455 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:50.186947 18455 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:50.187049 18455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:50.188108 18455 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf7d994a4824a538bca4fe6e20717b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34367 } }
I20260811 03:01:50.188465 18455 leader_election.cc:302] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcf7d994a4824a538bca4fe6e20717b4; no voters:
I20260811 03:01:50.188745 18455 leader_election.cc:288] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:50.188830 18469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:50.189031 18469 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [term 1 LEADER]: Becoming Leader. State: Replica: fcf7d994a4824a538bca4fe6e20717b4, State: Running, Role: LEADER
I20260811 03:01:50.189383 18469 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf7d994a4824a538bca4fe6e20717b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34367 } }
I20260811 03:01:50.189952 18455 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:50.190968 18470 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcf7d994a4824a538bca4fe6e20717b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf7d994a4824a538bca4fe6e20717b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34367 } } }
I20260811 03:01:50.191097 18470 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:50.191349 18472 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcf7d994a4824a538bca4fe6e20717b4. Latest consensus state: current_term: 1 leader_uuid: "fcf7d994a4824a538bca4fe6e20717b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf7d994a4824a538bca4fe6e20717b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34367 } } }
I20260811 03:01:50.191483 18488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:50.191493 18472 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:50.194574 18488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:50.200570 18488 catalog_manager.cc:1347] Generated new cluster ID: ae95534b4dec43eeb9f89cf91c20ee34
I20260811 03:01:50.200644 18488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:50.222635 18488 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:50.222950 18488 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:50.229928 18488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcf7d994a4824a538bca4fe6e20717b4: Generated new TSK 0
I20260811 03:01:50.230580 18488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:50.240906 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18312
2026-08-11T03:01:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1562 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:50 dist-test-slave-gjw7 krb5kdc[18505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:50 dist-test-slave-gjw7 krb5kdc[18505](info): set up 2 sockets
Aug 11 03:01:50 dist-test-slave-gjw7 krb5kdc[18505](info): commencing operation
krb5kdc: starting...
W20260811 03:01:51.440568 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:51 dist-test-slave-gjw7 krb5kdc[18505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:51.558246 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36433
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:51.668031 18526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:51.668272 18526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:51.668315 18526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:51.672190 18526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:51.672240 18526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:51.672259 18526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:51.672289 18526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:51.672314 18526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:51.677408 18526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:51.678367 18526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:51.679344 18526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:51.687466 18544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:51.687497 18548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:51.687510 18543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:51.687667 18526 server_base.cc:1034] running on GCE node
I20260811 03:01:51.688220 18526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:51.689239 18526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:51.690419 18526 hybrid_clock.cc:648] HybridClock initialized: now 1786417311690403 us; error 26 us; skew 500 ppm
Aug 11 03:01:51 dist-test-slave-gjw7 krb5kdc[18505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:51.692938 18526 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:51.693811 18526 webserver.cc:460] Webserver started at http://127.0.0.1:38543/ using document root <none> and password file <none>
I20260811 03:01:51.694320 18526 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:51.694398 18526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:51.694599 18526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:51.696542 18526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "eb8ccc8b34334960b60fb219df7a1ca5"
format_stamp: "Formatted at 2026-08-11 03:01:51 on dist-test-slave-gjw7"
server_key: "486ea7c7f45b142bbd417519a74a23fe"
server_key_iv: "a59c376408809a2044e3d18d047063c4"
server_key_version: "encryptionkey@0"
I20260811 03:01:51.697054 18526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "eb8ccc8b34334960b60fb219df7a1ca5"
format_stamp: "Formatted at 2026-08-11 03:01:51 on dist-test-slave-gjw7"
server_key: "486ea7c7f45b142bbd417519a74a23fe"
server_key_iv: "a59c376408809a2044e3d18d047063c4"
server_key_version: "encryptionkey@0"
I20260811 03:01:51.700501 18526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:01:51.702809 18560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:51.703924 18526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:51.704043 18526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "eb8ccc8b34334960b60fb219df7a1ca5"
format_stamp: "Formatted at 2026-08-11 03:01:51 on dist-test-slave-gjw7"
server_key: "486ea7c7f45b142bbd417519a74a23fe"
server_key_iv: "a59c376408809a2044e3d18d047063c4"
server_key_version: "encryptionkey@0"
I20260811 03:01:51.704131 18526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:51.723832 18526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:51.726743 18526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:51.726908 18526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:51.733273 18526 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36433
I20260811 03:01:51.733342 18655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36433 every 8 connection(s)
I20260811 03:01:51.734246 18526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:51.736884 18656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:51.742261 18656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5: Bootstrap starting.
I20260811 03:01:51.744004 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18526
I20260811 03:01:51.744127 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:51.744379 9005 external_mini_cluster.cc:1428] Setting key 62448dedde713e01976b5f338d6009d4
I20260811 03:01:51.744640 18656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:51.745515 18656 log.cc:824] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:51.747468 18656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5: No bootstrap required, opened a new log
I20260811 03:01:51.751432 18656 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260811 03:01:51.751718 18656 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:51.751780 18656 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb8ccc8b34334960b60fb219df7a1ca5, State: Initialized, Role: FOLLOWER
I20260811 03:01:51.752283 18656 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260811 03:01:51.752485 18656 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:51.752547 18656 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:51.752645 18656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 03:01:51 dist-test-slave-gjw7 krb5kdc[18505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 03:01:51.753612 18656 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260811 03:01:51.753983 18656 leader_election.cc:302] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb8ccc8b34334960b60fb219df7a1ca5; no voters:
I20260811 03:01:51.754330 18656 leader_election.cc:288] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:51.754725 18665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:51.754964 18665 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [term 1 LEADER]: Becoming Leader. State: Replica: eb8ccc8b34334960b60fb219df7a1ca5, State: Running, Role: LEADER
I20260811 03:01:51.755350 18665 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } }
I20260811 03:01:51.755602 18656 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:51.757068 18669 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb8ccc8b34334960b60fb219df7a1ca5. Latest consensus state: current_term: 1 leader_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } } }
I20260811 03:01:51.757220 18669 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:51.757545 18667 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8ccc8b34334960b60fb219df7a1ca5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36433 } } }
I20260811 03:01:51.757637 18667 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:51.757628 18687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:51.759780 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:51.745437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:37447 (local address 127.0.0.1:36433)
0811 03:01:51.745816 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:51.745832 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:51.745876 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:01:51.746610 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:51.746631 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:51.746963 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:51.747236 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:51.748123 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:51.749070 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:51.750254 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:51.750707 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:51.750819 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:51.753585 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:51.753592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:51.753610 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:51.753642 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:51.755732 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:51.756385 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:51.756392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:51.756398 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:51.756547 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:51.756886 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:51.756889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:51.756891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:51.757418 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:51.758380 (+ 962us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:51.758578 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 03:01:51.758808 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
I20260811 03:01:51.761967 18687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:51.762743 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:51.767969 18687 catalog_manager.cc:1347] Generated new cluster ID: f557422ec9814606adafb013c65b477a
I20260811 03:01:51.768031 18687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:51.772552 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:51.764700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:33259 (local address 127.0.0.1:36433)
0811 03:01:51.764806 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:51.764810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:51.764987 (+ 177us) server_negotiation.cc:407] Connection header received
0811 03:01:51.765088 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:51.765091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:51.765137 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:51.765218 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:51.765958 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:51.766599 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:51.767376 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:51.767617 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:51.767710 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:51.769326 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:51.769330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:51.769333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:51.769360 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:51.770881 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:51.771360 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:51.771363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:51.771365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:51.771452 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:51.771743 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:51.771746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:51.771748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:51.771944 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:51.772082 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:51.772312 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 03:01:51.772409 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:01:51.783535 18687 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:51.784650 18687 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:51.794188 18687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb8ccc8b34334960b60fb219df7a1ca5: Generated new TSK 0
I20260811 03:01:51.794929 18687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:51.798053 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18526
W20260811 03:01:51.816123 18503 connection.cc:535] client connection to 127.0.0.1:36433 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36433): Connection reset by peer (error 104)
2026-08-11T03:01:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1558 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:51 dist-test-slave-gjw7 krb5kdc[18707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:51 dist-test-slave-gjw7 krb5kdc[18707](info): set up 2 sockets
Aug 11 03:01:51 dist-test-slave-gjw7 krb5kdc[18707](info): commencing operation
krb5kdc: starting...
W20260811 03:01:53.008404 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:53 dist-test-slave-gjw7 krb5kdc[18707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:53.126431 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:53.234647 18741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:53.234910 18741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:53.234952 18741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:53.238868 18741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:53.238925 18741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:53.238947 18741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:53.238971 18741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:53.238989 18741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:53.244302 18741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:53.245226 18741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:53.246253 18741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:53.254345 18760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:53.254390 18754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:53.254457 18755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:53.254757 18741 server_base.cc:1034] running on GCE node
I20260811 03:01:53.255337 18741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:53.256434 18741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:53.257615 18741 hybrid_clock.cc:648] HybridClock initialized: now 1786417313257593 us; error 28 us; skew 500 ppm
Aug 11 03:01:53 dist-test-slave-gjw7 krb5kdc[18707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417313, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:53.263094 18741 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:53.263953 18741 webserver.cc:460] Webserver started at http://127.0.0.1:46735/ using document root <none> and password file <none>
I20260811 03:01:53.264467 18741 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:53.264532 18741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:53.264736 18741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:53.266851 18741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "091763bb62874239a208d8edc2e9ae97"
format_stamp: "Formatted at 2026-08-11 03:01:53 on dist-test-slave-gjw7"
server_key: "1d331280890f2d110f66dcea23df3d55"
server_key_iv: "16212d11c01ca2c9cc0cea8d1bd94aff"
server_key_version: "encryptionkey@0"
I20260811 03:01:53.267469 18741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "091763bb62874239a208d8edc2e9ae97"
format_stamp: "Formatted at 2026-08-11 03:01:53 on dist-test-slave-gjw7"
server_key: "1d331280890f2d110f66dcea23df3d55"
server_key_iv: "16212d11c01ca2c9cc0cea8d1bd94aff"
server_key_version: "encryptionkey@0"
I20260811 03:01:53.270988 18741 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:01:53.273177 18771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:53.274351 18741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:53.274459 18741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "091763bb62874239a208d8edc2e9ae97"
format_stamp: "Formatted at 2026-08-11 03:01:53 on dist-test-slave-gjw7"
server_key: "1d331280890f2d110f66dcea23df3d55"
server_key_iv: "16212d11c01ca2c9cc0cea8d1bd94aff"
server_key_version: "encryptionkey@0"
I20260811 03:01:53.274546 18741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:53.293220 18741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:53.295660 18741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:53.295810 18741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:53.302536 18741 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35233
I20260811 03:01:53.302567 18875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35233 every 8 connection(s)
I20260811 03:01:53.303464 18741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:53.306234 18877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:53.311424 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18741
I20260811 03:01:53.311558 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:53.311820 18877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97: Bootstrap starting.
I20260811 03:01:53.311875 9005 external_mini_cluster.cc:1428] Setting key 371938aaa325073b254cf6c009f5177f
I20260811 03:01:53.314317 18877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:53.315129 18877 log.cc:824] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:53.317154 18877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97: No bootstrap required, opened a new log
I20260811 03:01:53.320395 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:53.313103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:44175 (local address 127.0.0.1:35233)
0811 03:01:53.313528 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:53.313544 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:53.313580 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:01:53.314193 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:53.314211 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:53.314589 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:53.314879 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:53.315193 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:53.316182 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:53.317401 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:53.317862 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:53.317992 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:53.318815 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:53.318821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:53.318833 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:53.319203 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:53.319254 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:53.319277 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:53.319369 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:53.319551 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 03:01:53.319675 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":120,"threads_started":1}
I20260811 03:01:53.320983 18877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091763bb62874239a208d8edc2e9ae97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260811 03:01:53.321192 18877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:53.321239 18877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091763bb62874239a208d8edc2e9ae97, State: Initialized, Role: FOLLOWER
I20260811 03:01:53.321714 18877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091763bb62874239a208d8edc2e9ae97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260811 03:01:53.321859 18877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:53.321923 18877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:53.322024 18877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:53.322875 18877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091763bb62874239a208d8edc2e9ae97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260811 03:01:53.323024 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:53.323204 18877 leader_election.cc:302] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 091763bb62874239a208d8edc2e9ae97; no voters:
I20260811 03:01:53.323474 18877 leader_election.cc:288] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:53.323597 18884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:53.323832 18884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [term 1 LEADER]: Becoming Leader. State: Replica: 091763bb62874239a208d8edc2e9ae97, State: Running, Role: LEADER
I20260811 03:01:53.324138 18884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091763bb62874239a208d8edc2e9ae97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } }
I20260811 03:01:53.324667 18877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:53.325790 18885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "091763bb62874239a208d8edc2e9ae97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091763bb62874239a208d8edc2e9ae97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } } }
I20260811 03:01:53.325914 18885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:53.326143 18886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 091763bb62874239a208d8edc2e9ae97. Latest consensus state: current_term: 1 leader_uuid: "091763bb62874239a208d8edc2e9ae97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091763bb62874239a208d8edc2e9ae97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35233 } } }
I20260811 03:01:53.326242 18886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:53.329672 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:53.325173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:58753 (local address 127.0.0.1:35233)
0811 03:01:53.325265 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:53.325269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:53.325478 (+ 209us) server_negotiation.cc:407] Connection header received
0811 03:01:53.325663 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:53.325666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:53.325711 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:53.325760 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:53.326344 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:53.327056 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:53.327950 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:53.328262 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:53.328374 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:53.329230 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:53.329233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:53.329236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:53.329250 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:53.329279 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:53.329315 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:53.329405 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:53.329522 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 03:01:53.329555 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 03:01:53.330003 18925 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:01:53.330070 18925 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:01:53.330135 18909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:53.330952 18909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:53.335522 18909 catalog_manager.cc:1347] Generated new cluster ID: 6e28f6be507c462295de8606f8dc976f
I20260811 03:01:53.335582 18909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:53.359131 18909 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:53.359997 18909 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:53.363989 18909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 091763bb62874239a208d8edc2e9ae97: Generated new TSK 0
I20260811 03:01:53.364677 18909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:53.371795 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18741
W20260811 03:01:53.392174 18705 connection.cc:535] client connection to 127.0.0.1:35233 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35233): Connection reset by peer (error 104)
2026-08-11T03:01:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:53 dist-test-slave-gjw7 krb5kdc[18940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:53 dist-test-slave-gjw7 krb5kdc[18940](info): set up 2 sockets
Aug 11 03:01:53 dist-test-slave-gjw7 krb5kdc[18940](info): commencing operation
krb5kdc: starting...
W20260811 03:01:54.618140 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:54 dist-test-slave-gjw7 krb5kdc[18940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 03:01:54.739208 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:54.850975 18960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:54.851270 18960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:54.851325 18960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:54.855296 18960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:54.855353 18960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:54.855379 18960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:54.855417 18960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:54.855448 18960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:54.860534 18960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36255
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:54.861663 18960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:54.862709 18960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:54.870854 18977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:54.870931 18980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:54.870931 18983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:54.871228 18960 server_base.cc:1034] running on GCE node
I20260811 03:01:54.871817 18960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:54.872959 18960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:54.874137 18960 hybrid_clock.cc:648] HybridClock initialized: now 1786417314874095 us; error 50 us; skew 500 ppm
Aug 11 03:01:54 dist-test-slave-gjw7 krb5kdc[18940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:54.876752 18960 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 03:01:54.877629 18960 webserver.cc:460] Webserver started at http://127.0.0.1:34169/ using document root <none> and password file <none>
I20260811 03:01:54.878123 18960 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:54.878177 18960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:54.878340 18960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:54.880316 18960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "68f5493959074a38a2a22646a7323905"
format_stamp: "Formatted at 2026-08-11 03:01:54 on dist-test-slave-gjw7"
server_key: "1af7cc20649173e48ba0602f83d5efb3"
server_key_iv: "9dda4a6885fa9b2a2d7e00f19ca4359d"
server_key_version: "encryptionkey@0"
I20260811 03:01:54.880820 18960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "68f5493959074a38a2a22646a7323905"
format_stamp: "Formatted at 2026-08-11 03:01:54 on dist-test-slave-gjw7"
server_key: "1af7cc20649173e48ba0602f83d5efb3"
server_key_iv: "9dda4a6885fa9b2a2d7e00f19ca4359d"
server_key_version: "encryptionkey@0"
I20260811 03:01:54.884274 18960 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:01:54.886534 18993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:54.887677 18960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:54.887785 18960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "68f5493959074a38a2a22646a7323905"
format_stamp: "Formatted at 2026-08-11 03:01:54 on dist-test-slave-gjw7"
server_key: "1af7cc20649173e48ba0602f83d5efb3"
server_key_iv: "9dda4a6885fa9b2a2d7e00f19ca4359d"
server_key_version: "encryptionkey@0"
I20260811 03:01:54.887872 18960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:54.892163 18960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:54.894846 18960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:54.895015 18960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:54.901239 18960 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36255
I20260811 03:01:54.901347 19102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36255 every 8 connection(s)
I20260811 03:01:54.902145 18960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:54.904301 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 18960
I20260811 03:01:54.904422 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:54.904681 9005 external_mini_cluster.cc:1428] Setting key 30dde60a4ebb59cea18a4a05a9ffc599
I20260811 03:01:54.904814 19103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:54.910018 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:54.905914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:33999 (local address 127.0.0.1:36255)
0811 03:01:54.906293 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:54.906308 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:54.906604 (+ 296us) server_negotiation.cc:407] Connection header received
0811 03:01:54.907604 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:54.907623 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:54.907899 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:54.908196 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:54.908919 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:54.908924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:54.908936 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:54.908946 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:54.908997 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:54.909016 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:54.909138 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:54.909214 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 03:01:54.909391 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
I20260811 03:01:54.911026 19103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905: Bootstrap starting.
I20260811 03:01:54.912676 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:54.913403 19103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:54.914122 19103 log.cc:824] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:54.916129 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:54.914632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:59057 (local address 127.0.0.1:36255)
0811 03:01:54.914730 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:54.914734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:54.914956 (+ 222us) server_negotiation.cc:407] Connection header received
0811 03:01:54.915076 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:54.915078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:54.915106 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:54.915157 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:54.915781 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:54.915783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:54.915785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:54.915795 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:54.915819 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:54.915832 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:54.915897 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:54.916000 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 03:01:54.916031 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 03:01:54.916335 19103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905: No bootstrap required, opened a new log
I20260811 03:01:54.919760 19103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f5493959074a38a2a22646a7323905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36255 } }
I20260811 03:01:54.919976 19103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:54.920028 19103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f5493959074a38a2a22646a7323905, State: Initialized, Role: FOLLOWER
I20260811 03:01:54.920457 19103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f5493959074a38a2a22646a7323905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36255 } }
I20260811 03:01:54.920607 19103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:54.920670 19103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:54.920773 19103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:54.921659 19103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f5493959074a38a2a22646a7323905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36255 } }
I20260811 03:01:54.921972 19103 leader_election.cc:302] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68f5493959074a38a2a22646a7323905; no voters:
I20260811 03:01:54.922228 19103 leader_election.cc:288] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:54.922340 19121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:54.922551 19121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [term 1 LEADER]: Becoming Leader. State: Replica: 68f5493959074a38a2a22646a7323905, State: Running, Role: LEADER
I20260811 03:01:54.922833 19121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f5493959074a38a2a22646a7323905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36255 } }
I20260811 03:01:54.923310 19103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:54.924481 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68f5493959074a38a2a22646a7323905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f5493959074a38a2a22646a7323905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36255 } } }
I20260811 03:01:54.924494 19124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68f5493959074a38a2a22646a7323905. Latest consensus state: current_term: 1 leader_uuid: "68f5493959074a38a2a22646a7323905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68f5493959074a38a2a22646a7323905" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36255 } } }
I20260811 03:01:54.924607 19124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:54.924606 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:54.925577 19135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:54.928376 19135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:54.933775 19135 catalog_manager.cc:1347] Generated new cluster ID: 592272e201bc46ccad7f932ab56d87d1
I20260811 03:01:54.933837 19135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:54.955608 19135 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:54.955968 19135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:54.967602 19135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68f5493959074a38a2a22646a7323905: Generated new TSK 0
I20260811 03:01:54.968488 19135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:54.982921 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 18960
2026-08-11T03:01:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:55 dist-test-slave-gjw7 krb5kdc[19157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:55 dist-test-slave-gjw7 krb5kdc[19157](info): set up 2 sockets
Aug 11 03:01:55 dist-test-slave-gjw7 krb5kdc[19157](info): commencing operation
krb5kdc: starting...
W20260811 03:01:56.195087 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:56 dist-test-slave-gjw7 krb5kdc[19157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:01:56.310158 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:33207
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:33207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:56.418543 19180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:56.418792 19180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:56.418833 19180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:56.422662 19180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:56.422712 19180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:56.422731 19180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:56.422751 19180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:56.422765 19180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:56.427714 19180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:33207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:56.428602 19180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:56.429529 19180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:56.437549 19190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:56.437624 19196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:56.437572 19191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:56.437608 19180 server_base.cc:1034] running on GCE node
I20260811 03:01:56.438325 19180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:56.439327 19180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:56.440490 19180 hybrid_clock.cc:648] HybridClock initialized: now 1786417316440476 us; error 25 us; skew 500 ppm
Aug 11 03:01:56 dist-test-slave-gjw7 krb5kdc[19157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:56.443043 19180 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:01:56.443840 19180 webserver.cc:460] Webserver started at http://127.8.203.126:43085/ using document root <none> and password file <none>
I20260811 03:01:56.444347 19180 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:56.444411 19180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:56.444613 19180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:56.446617 19180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "063c125a63b44b3b98c7dca5c8a4af11"
format_stamp: "Formatted at 2026-08-11 03:01:56 on dist-test-slave-gjw7"
server_key: "34fe22a4c9455d0743c4dcfe491f47e5"
server_key_iv: "485eda5a09cf620a8ee1ca042ce7e000"
server_key_version: "encryptionkey@0"
I20260811 03:01:56.447147 19180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "063c125a63b44b3b98c7dca5c8a4af11"
format_stamp: "Formatted at 2026-08-11 03:01:56 on dist-test-slave-gjw7"
server_key: "34fe22a4c9455d0743c4dcfe491f47e5"
server_key_iv: "485eda5a09cf620a8ee1ca042ce7e000"
server_key_version: "encryptionkey@0"
I20260811 03:01:56.450632 19180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:01:56.452783 19209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:56.453933 19180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:56.454051 19180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "063c125a63b44b3b98c7dca5c8a4af11"
format_stamp: "Formatted at 2026-08-11 03:01:56 on dist-test-slave-gjw7"
server_key: "34fe22a4c9455d0743c4dcfe491f47e5"
server_key_iv: "485eda5a09cf620a8ee1ca042ce7e000"
server_key_version: "encryptionkey@0"
I20260811 03:01:56.454139 19180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:56.470237 19180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:56.473153 19180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:56.473347 19180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:56.479826 19180 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:33207
I20260811 03:01:56.479863 19315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:33207 every 8 connection(s)
I20260811 03:01:56.480729 19180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:56.483414 19317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:56.485507 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19180
I20260811 03:01:56.485637 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:56.485910 9005 external_mini_cluster.cc:1428] Setting key 1ed4088ee36f772d69eef6d463356dcf
I20260811 03:01:56.489569 19317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11: Bootstrap starting.
I20260811 03:01:56.492111 19317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:56.492822 19317 log.cc:824] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:56.494729 19317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11: No bootstrap required, opened a new log
Aug 11 03:01:56 dist-test-slave-gjw7 krb5kdc[19157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:01:56.498522 19317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063c125a63b44b3b98c7dca5c8a4af11" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33207 } }
I20260811 03:01:56.498785 19317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:56.498860 19317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 063c125a63b44b3b98c7dca5c8a4af11, State: Initialized, Role: FOLLOWER
I20260811 03:01:56.499280 19317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063c125a63b44b3b98c7dca5c8a4af11" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33207 } }
I20260811 03:01:56.499398 19317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:56.499483 19317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:56.499579 19317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:56.500494 19317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063c125a63b44b3b98c7dca5c8a4af11" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33207 } }
I20260811 03:01:56.500805 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:56.487242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33329 (local address 127.8.203.126:33207)
0811 03:01:56.487671 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:56.487690 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:56.487734 (+ 44us) server_negotiation.cc:407] Connection header received
0811 03:01:56.488519 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:56.488543 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:56.488962 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:56.489336 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:56.490026 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:56.491479 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:56.492318 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:56.492794 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:56.492952 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:56.495523 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:56.495529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:56.495545 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:56.495575 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:56.497941 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:56.498412 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:56.498418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:56.498424 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:56.498563 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:56.498858 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:56.498862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:56.498864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:56.499312 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:56.499527 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:56.499856 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 03:01:56.500066 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":110,"threads_started":1}
I20260811 03:01:56.500804 19317 leader_election.cc:302] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 063c125a63b44b3b98c7dca5c8a4af11; no voters:
I20260811 03:01:56.501163 19317 leader_election.cc:288] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:56.501233 19326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:56.501464 19326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [term 1 LEADER]: Becoming Leader. State: Replica: 063c125a63b44b3b98c7dca5c8a4af11, State: Running, Role: LEADER
I20260811 03:01:56.501804 19326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063c125a63b44b3b98c7dca5c8a4af11" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33207 } }
I20260811 03:01:56.502456 19317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:56.503736 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:56.504158 19331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "063c125a63b44b3b98c7dca5c8a4af11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063c125a63b44b3b98c7dca5c8a4af11" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33207 } } }
I20260811 03:01:56.504158 19328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 063c125a63b44b3b98c7dca5c8a4af11. Latest consensus state: current_term: 1 leader_uuid: "063c125a63b44b3b98c7dca5c8a4af11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063c125a63b44b3b98c7dca5c8a4af11" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 33207 } } }
I20260811 03:01:56.504312 19331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:56.504312 19328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11 [sys.catalog]: This master's current role is: LEADER
W20260811 03:01:56.507529 19354 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:01:56.507596 19354 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:01:56.507659 19352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:56.508466 19352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:56.514101 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:56.505902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55251 (local address 127.8.203.126:33207)
0811 03:01:56.506005 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:56.506009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:56.506173 (+ 164us) server_negotiation.cc:407] Connection header received
0811 03:01:56.506271 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:56.506274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:56.506314 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:56.506398 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:56.507126 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:56.507854 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:56.508929 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:56.509266 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:56.509402 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:56.510823 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:56.510827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:56.510829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:01:56.510852 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:56.512236 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:56.512772 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:56.512776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:56.512778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:56.512878 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:01:56.513215 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:01:56.513219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:01:56.513220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:01:56.513451 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:56.513738 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:56.513869 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 03:01:56.513966 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:01:56.514390 19352 catalog_manager.cc:1347] Generated new cluster ID: ecce04752e144494ae36fe39b6c2ee3c
I20260811 03:01:56.514442 19352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:56.534862 19352 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:56.535682 19352 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:56.544157 19352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 063c125a63b44b3b98c7dca5c8a4af11: Generated new TSK 0
I20260811 03:01:56.544749 19352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:56.550055 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19180
W20260811 03:01:56.565652 19153 connection.cc:535] client connection to 127.8.203.126:33207 recv error: Network error: failed to read from TLS socket (remote: 127.8.203.126:33207): Connection reset by peer (error 104)
2026-08-11T03:01:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1563 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:56 dist-test-slave-gjw7 krb5kdc[19368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:56 dist-test-slave-gjw7 krb5kdc[19368](info): set up 2 sockets
Aug 11 03:01:56 dist-test-slave-gjw7 krb5kdc[19368](info): commencing operation
krb5kdc: starting...
W20260811 03:01:57.782444 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:57 dist-test-slave-gjw7 krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:01:57.900017 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36865
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:36865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:58.007915 19405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:58.008136 19405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:58.008178 19405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:58.012036 19405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:58.012086 19405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:58.012106 19405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:58.012137 19405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:58.012162 19405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:58.017243 19405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:36865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:58.018153 19405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:58.019071 19405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:58.027083 19417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:58.027100 19420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:58.027148 19405 server_base.cc:1034] running on GCE node
W20260811 03:01:58.027084 19410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:58.027961 19405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:58.028964 19405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:58.030112 19405 hybrid_clock.cc:648] HybridClock initialized: now 1786417318030094 us; error 31 us; skew 500 ppm
Aug 11 03:01:58 dist-test-slave-gjw7 krb5kdc[19368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417318, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:58.032840 19405 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:01:58.033732 19405 webserver.cc:460] Webserver started at http://127.8.203.126:45729/ using document root <none> and password file <none>
I20260811 03:01:58.034255 19405 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:58.034317 19405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:58.034512 19405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:58.036454 19405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "be429cf4d6a8457789d3dfd3c927a1a6"
format_stamp: "Formatted at 2026-08-11 03:01:58 on dist-test-slave-gjw7"
server_key: "097fc4e22e3f9a362d60abb95392201c"
server_key_iv: "2000a8e3224fef20b8dc71b694a41e92"
server_key_version: "encryptionkey@0"
I20260811 03:01:58.036981 19405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "be429cf4d6a8457789d3dfd3c927a1a6"
format_stamp: "Formatted at 2026-08-11 03:01:58 on dist-test-slave-gjw7"
server_key: "097fc4e22e3f9a362d60abb95392201c"
server_key_iv: "2000a8e3224fef20b8dc71b694a41e92"
server_key_version: "encryptionkey@0"
I20260811 03:01:58.040416 19405 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:01:58.042810 19437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:58.043967 19405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:01:58.044101 19405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "be429cf4d6a8457789d3dfd3c927a1a6"
format_stamp: "Formatted at 2026-08-11 03:01:58 on dist-test-slave-gjw7"
server_key: "097fc4e22e3f9a362d60abb95392201c"
server_key_iv: "2000a8e3224fef20b8dc71b694a41e92"
server_key_version: "encryptionkey@0"
I20260811 03:01:58.044205 19405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:58.054358 19405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:58.056954 19405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:58.057129 19405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:58.064214 19405 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:36865
I20260811 03:01:58.064218 19546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:36865 every 8 connection(s)
I20260811 03:01:58.065164 19405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:58.065510 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19405
I20260811 03:01:58.065621 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:58.065936 9005 external_mini_cluster.cc:1428] Setting key 2355eec80415b01c074a819379b80a36
I20260811 03:01:58.068254 19548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:58.074371 19548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6: Bootstrap starting.
I20260811 03:01:58.074826 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:58.067238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54141 (local address 127.8.203.126:36865)
0811 03:01:58.067652 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:58.067676 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:58.067713 (+ 37us) server_negotiation.cc:407] Connection header received
0811 03:01:58.068396 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:58.068416 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:58.068758 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:58.069039 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:58.069403 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:58.070386 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:58.071598 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:58.072063 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:58.072180 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:58.073029 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:58.073034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:58.073046 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:58.073460 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:58.073515 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:58.073537 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:58.073637 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:58.073829 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 03:01:58.073969 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":122,"threads_started":1}
I20260811 03:01:58.076815 19548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:58.077677 19548 log.cc:824] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:58.077811 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:58.079841 19548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6: No bootstrap required, opened a new log
I20260811 03:01:58.083346 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:58.079908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48549 (local address 127.8.203.126:36865)
0811 03:01:58.080027 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:58.080031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:58.080152 (+ 121us) server_negotiation.cc:407] Connection header received
0811 03:01:58.080252 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:58.080255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:58.080294 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:58.080355 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:58.080600 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:58.081099 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:58.081910 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:01:58.082117 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:01:58.082199 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:01:58.082928 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:58.082932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:58.082935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:58.082946 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:58.082969 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:58.082979 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:58.083043 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:58.083189 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 03:01:58.083223 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 03:01:58.083180 19548 raft_consensus.cc:348] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36865 } }
I20260811 03:01:58.083436 19548 raft_consensus.cc:374] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:58.083510 19548 raft_consensus.cc:729] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be429cf4d6a8457789d3dfd3c927a1a6, State: Initialized, Role: FOLLOWER
I20260811 03:01:58.083961 19548 consensus_queue.cc:260] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36865 } }
I20260811 03:01:58.084122 19548 raft_consensus.cc:388] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:58.084195 19548 raft_consensus.cc:482] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:58.084291 19548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:58.085106 19548 raft_consensus.cc:504] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36865 } }
I20260811 03:01:58.085443 19548 leader_election.cc:302] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be429cf4d6a8457789d3dfd3c927a1a6; no voters:
I20260811 03:01:58.085709 19548 leader_election.cc:288] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:58.085798 19565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:58.085999 19565 raft_consensus.cc:686] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [term 1 LEADER]: Becoming Leader. State: Replica: be429cf4d6a8457789d3dfd3c927a1a6, State: Running, Role: LEADER
I20260811 03:01:58.086305 19565 consensus_queue.cc:237] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36865 } }
I20260811 03:01:58.086805 19548 sys_catalog.cc:564] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:58.087746 19567 sys_catalog.cc:455] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be429cf4d6a8457789d3dfd3c927a1a6. Latest consensus state: current_term: 1 leader_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36865 } } }
I20260811 03:01:58.087880 19567 sys_catalog.cc:458] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:58.088259 19580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:58.088220 19566 sys_catalog.cc:455] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be429cf4d6a8457789d3dfd3c927a1a6" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36865 } } }
I20260811 03:01:58.088366 19566 sys_catalog.cc:458] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:58.091149 19580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:58.096736 19580 catalog_manager.cc:1347] Generated new cluster ID: a3e9371648594067b315634bcb3f66b7
I20260811 03:01:58.096796 19580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:58.107542 19580 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:58.108482 19580 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:58.114140 19580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be429cf4d6a8457789d3dfd3c927a1a6: Generated new TSK 0
I20260811 03:01:58.114722 19580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:58.127280 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19405
W20260811 03:01:58.142829 19366 connection.cc:535] client connection to 127.8.203.126:36865 recv error: Network error: failed to read from TLS socket (remote: 127.8.203.126:36865): Connection reset by peer (error 104)
2026-08-11T03:01:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:58 dist-test-slave-gjw7 krb5kdc[19601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:58 dist-test-slave-gjw7 krb5kdc[19601](info): set up 2 sockets
Aug 11 03:01:58 dist-test-slave-gjw7 krb5kdc[19601](info): commencing operation
krb5kdc: starting...
W20260811 03:01:59.341882 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:01:59 dist-test-slave-gjw7 krb5kdc[19601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:01:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:01:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:01:59.464699 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41405
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:41405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:01:59.572546 19623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:01:59.572793 19623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:01:59.572847 19623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:01:59.576748 19623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:01:59.576807 19623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:01:59.576833 19623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:01:59.576854 19623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:01:59.576874 19623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:01:59.581929 19623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:41405
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:41405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:01:59.582887 19623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:01:59.583837 19623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:01:59.591802 19635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:59.591909 19623 server_base.cc:1034] running on GCE node
W20260811 03:01:59.591881 19632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:01:59.591804 19643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:01:59.592562 19623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:01:59.595837 19623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:01:59.597016 19623 hybrid_clock.cc:648] HybridClock initialized: now 1786417319596953 us; error 80 us; skew 500 ppm
Aug 11 03:01:59 dist-test-slave-gjw7 krb5kdc[19601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417319, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:01:59.599613 19623 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:01:59.600464 19623 webserver.cc:460] Webserver started at http://127.8.203.126:33907/ using document root <none> and password file <none>
I20260811 03:01:59.600975 19623 fs_manager.cc:359] Metadata directory not provided
I20260811 03:01:59.601034 19623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:01:59.601228 19623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:01:59.603199 19623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "89a2701afcf9416bbbb519e14b2ad170"
format_stamp: "Formatted at 2026-08-11 03:01:59 on dist-test-slave-gjw7"
server_key: "6f200e273f42a9b02956d8f53efef2bb"
server_key_iv: "31d108939bf00ca6ddbe0e6f7fef152f"
server_key_version: "encryptionkey@0"
I20260811 03:01:59.603740 19623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "89a2701afcf9416bbbb519e14b2ad170"
format_stamp: "Formatted at 2026-08-11 03:01:59 on dist-test-slave-gjw7"
server_key: "6f200e273f42a9b02956d8f53efef2bb"
server_key_iv: "31d108939bf00ca6ddbe0e6f7fef152f"
server_key_version: "encryptionkey@0"
I20260811 03:01:59.607306 19623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 03:01:59.609521 19656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:01:59.610628 19623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:01:59.610741 19623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "89a2701afcf9416bbbb519e14b2ad170"
format_stamp: "Formatted at 2026-08-11 03:01:59 on dist-test-slave-gjw7"
server_key: "6f200e273f42a9b02956d8f53efef2bb"
server_key_iv: "31d108939bf00ca6ddbe0e6f7fef152f"
server_key_version: "encryptionkey@0"
I20260811 03:01:59.610836 19623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:01:59.615159 19623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:01:59.617790 19623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:01:59.617949 19623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:01:59.623971 19623 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:41405
I20260811 03:01:59.624058 19755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:41405 every 8 connection(s)
I20260811 03:01:59.624816 19623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:01:59.627416 19756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:01:59.629791 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19623
I20260811 03:01:59.629927 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:01:59.630168 9005 external_mini_cluster.cc:1428] Setting key 450a240d1568839a037cf2df14d4d891
I20260811 03:01:59.632861 19756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170: Bootstrap starting.
I20260811 03:01:59.634935 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:59.631366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48615 (local address 127.8.203.126:41405)
0811 03:01:59.631762 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:59.631777 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:59.631813 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:01:59.632430 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:59.632447 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:59.632717 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:59.633012 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:59.633795 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:59.633799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:59.633811 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:59.633820 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:59.633873 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:59.633893 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:59.633992 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:59.634105 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 03:01:59.634241 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":126,"threads_started":1}
I20260811 03:01:59.635329 19756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170: Neither blocks nor log segments found. Creating new log.
I20260811 03:01:59.636133 19756 log.cc:824] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170: Log is configured to *not* fsync() on all Append() calls
I20260811 03:01:59.637786 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:01:59.638108 19756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170: No bootstrap required, opened a new log
I20260811 03:01:59.641208 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:01:59.639703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38393 (local address 127.8.203.126:41405)
0811 03:01:59.639813 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 03:01:59.639817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:01:59.639942 (+ 125us) server_negotiation.cc:407] Connection header received
0811 03:01:59.640031 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:01:59.640034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:01:59.640068 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:01:59.640146 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:01:59.640846 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:01:59.640850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:01:59.640853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:01:59.640866 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:01:59.640907 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:01:59.640922 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:01:59.641012 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 03:01:59.641048 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 03:01:59.641085 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 03:01:59.642174 19756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a2701afcf9416bbbb519e14b2ad170" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 03:01:59.642398 19756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:01:59.642447 19756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a2701afcf9416bbbb519e14b2ad170, State: Initialized, Role: FOLLOWER
I20260811 03:01:59.642829 19756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a2701afcf9416bbbb519e14b2ad170" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 03:01:59.642952 19756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:01:59.642997 19756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:01:59.643067 19756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:01:59.643884 19756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a2701afcf9416bbbb519e14b2ad170" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 03:01:59.644177 19756 leader_election.cc:302] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89a2701afcf9416bbbb519e14b2ad170; no voters:
I20260811 03:01:59.644429 19756 leader_election.cc:288] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:01:59.644531 19780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:01:59.644802 19780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [term 1 LEADER]: Becoming Leader. State: Replica: 89a2701afcf9416bbbb519e14b2ad170, State: Running, Role: LEADER
I20260811 03:01:59.645169 19780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a2701afcf9416bbbb519e14b2ad170" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } }
I20260811 03:01:59.645654 19756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:01:59.646250 19780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89a2701afcf9416bbbb519e14b2ad170. Latest consensus state: current_term: 1 leader_uuid: "89a2701afcf9416bbbb519e14b2ad170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a2701afcf9416bbbb519e14b2ad170" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } } }
I20260811 03:01:59.646363 19780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:59.646732 19791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:01:59.646761 19785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89a2701afcf9416bbbb519e14b2ad170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a2701afcf9416bbbb519e14b2ad170" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 41405 } } }
I20260811 03:01:59.646963 19785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170 [sys.catalog]: This master's current role is: LEADER
I20260811 03:01:59.649807 19791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:01:59.654846 19791 catalog_manager.cc:1347] Generated new cluster ID: fb564a3c2ef24f8881652c1f585ddbc1
I20260811 03:01:59.654904 19791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:01:59.672346 19791 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:01:59.672612 19791 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:01:59.679813 19791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89a2701afcf9416bbbb519e14b2ad170: Generated new TSK 0
I20260811 03:01:59.680390 19791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:01:59.690963 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19623
2026-08-11T03:01:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:01:59 dist-test-slave-gjw7 krb5kdc[19814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:01:59 dist-test-slave-gjw7 krb5kdc[19814](info): set up 2 sockets
Aug 11 03:01:59 dist-test-slave-gjw7 krb5kdc[19814](info): commencing operation
krb5kdc: starting...
W20260811 03:02:00.895542 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:02:00 dist-test-slave-gjw7 krb5kdc[19814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:02:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:02:01.011879 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:44881
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:42413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:44881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:02:01.121394 19841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:01.121658 19841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:01.121726 19841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:01.125672 19841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:01.125735 19841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:01.125767 19841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:01.125792 19841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:01.125826 19841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:01.130959 19841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:42413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:44881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:44881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:01.131943 19841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:01.132929 19841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:02:01.141079 19851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:01.141079 19857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:01.141120 19841 server_base.cc:1034] running on GCE node
W20260811 03:02:01.141108 19853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:01.141863 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:01.142866 19841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:01.144026 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786417321144006 us; error 34 us; skew 500 ppm
Aug 11 03:02:01 dist-test-slave-gjw7 krb5kdc[19814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:01.146662 19841 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:02:01.147451 19841 webserver.cc:460] Webserver started at http://127.8.203.126:45081/ using document root <none> and password file <none>
I20260811 03:02:01.147938 19841 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:01.147993 19841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:01.148154 19841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:01.150117 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "7a2030162f6648f2857030d4a4cb0e58"
format_stamp: "Formatted at 2026-08-11 03:02:01 on dist-test-slave-gjw7"
server_key: "5ae33b241108ab3da75dbdcef7074930"
server_key_iv: "a1386765e48fcadde40f175927593c21"
server_key_version: "encryptionkey@0"
I20260811 03:02:01.150645 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "7a2030162f6648f2857030d4a4cb0e58"
format_stamp: "Formatted at 2026-08-11 03:02:01 on dist-test-slave-gjw7"
server_key: "5ae33b241108ab3da75dbdcef7074930"
server_key_iv: "a1386765e48fcadde40f175927593c21"
server_key_version: "encryptionkey@0"
I20260811 03:02:01.154096 19841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:02:01.156314 19869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:01.157542 19841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:02:01.157639 19841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "7a2030162f6648f2857030d4a4cb0e58"
format_stamp: "Formatted at 2026-08-11 03:02:01 on dist-test-slave-gjw7"
server_key: "5ae33b241108ab3da75dbdcef7074930"
server_key_iv: "a1386765e48fcadde40f175927593c21"
server_key_version: "encryptionkey@0"
I20260811 03:02:01.157720 19841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:01.172816 19841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:01.175418 19841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:01.175565 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:01.181926 19841 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:44881
I20260811 03:02:01.181965 19974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:44881 every 8 connection(s)
I20260811 03:02:01.182811 19841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:02:01.185546 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:01.187106 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 19841
I20260811 03:02:01.187216 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:02:01.187465 9005 external_mini_cluster.cc:1428] Setting key 70c9110e3b2281178d7797e4dd2d631a
I20260811 03:02:01.192021 19975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58: Bootstrap starting.
I20260811 03:02:01.194615 19975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:01.195477 19975 log.cc:824] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:02:01 dist-test-slave-gjw7 krb5kdc[19814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:01.198155 19975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58: No bootstrap required, opened a new log
I20260811 03:02:01.201918 19980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:01.188791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36389 (local address 127.8.203.126:44881)
0811 03:02:01.189226 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:01.189245 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:01.189311 (+ 66us) server_negotiation.cc:407] Connection header received
0811 03:02:01.190130 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:01.190155 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:01.190611 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:01.190987 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:01.191621 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:01.193062 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:01.193964 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:01.194633 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:01.194792 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:01.197097 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:01.197104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:01.197120 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:01.197151 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:01.198995 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:01.199377 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:01.199384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:01.199389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:01.199525 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:01.199732 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:01.199735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:01.199736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:01.200254 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:01.200457 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:01.200616 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 03:02:01.200926 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":112,"threads_started":1}
I20260811 03:02:01.202543 19975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2030162f6648f2857030d4a4cb0e58" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44881 } }
I20260811 03:02:01.202760 19975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:02:01.202817 19975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2030162f6648f2857030d4a4cb0e58, State: Initialized, Role: FOLLOWER
I20260811 03:02:01.203236 19975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2030162f6648f2857030d4a4cb0e58" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44881 } }
I20260811 03:02:01.203379 19975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:02:01.203446 19975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:02:01.203549 19975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:01.204541 19975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2030162f6648f2857030d4a4cb0e58" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44881 } }
I20260811 03:02:01.204600 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:02:01.204877 19975 leader_election.cc:302] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a2030162f6648f2857030d4a4cb0e58; no voters:
I20260811 03:02:01.205152 19975 leader_election.cc:288] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:02:01.205274 19983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:02:01.205540 19983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [term 1 LEADER]: Becoming Leader. State: Replica: 7a2030162f6648f2857030d4a4cb0e58, State: Running, Role: LEADER
I20260811 03:02:01.205919 19983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2030162f6648f2857030d4a4cb0e58" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44881 } }
I20260811 03:02:01.206372 19975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:01.207082 19983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a2030162f6648f2857030d4a4cb0e58. Latest consensus state: current_term: 1 leader_uuid: "7a2030162f6648f2857030d4a4cb0e58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2030162f6648f2857030d4a4cb0e58" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44881 } } }
I20260811 03:02:01.207194 19983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:01.207508 19986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a2030162f6648f2857030d4a4cb0e58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2030162f6648f2857030d4a4cb0e58" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 44881 } } }
I20260811 03:02:01.207609 19986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:01.208271 20012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:02:01.211406 20012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:02:01.216322 19980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:01.207897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33713 (local address 127.8.203.126:44881)
0811 03:02:01.208011 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:01.208015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:01.208665 (+ 650us) server_negotiation.cc:407] Connection header received
0811 03:02:01.208848 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:01.208851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:01.208894 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:01.208942 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:01.209781 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:01.210285 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:01.211061 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:01.211280 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:01.211365 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:01.212585 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:01.212588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:01.212590 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:01.212611 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:01.214122 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:01.214591 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:01.214594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:01.214596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:01.214684 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:01.214974 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:01.214978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:01.214980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:01.215210 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:01.215345 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:01.216050 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 03:02:01.216164 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 03:02:01.217200 20012 catalog_manager.cc:1347] Generated new cluster ID: de1a304d4f87453ab1cd2d7587a4c0a0
I20260811 03:02:01.217325 20012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:02:01.223670 20012 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:02:01.224485 20012 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:02:01.229027 20012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a2030162f6648f2857030d4a4cb0e58: Generated new TSK 0
I20260811 03:02:01.229662 20012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:02:01.235469 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 19841
2026-08-11T03:02:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:02:01 dist-test-slave-gjw7 krb5kdc[20036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:02:01 dist-test-slave-gjw7 krb5kdc[20036](info): set up 2 sockets
Aug 11 03:02:01 dist-test-slave-gjw7 krb5kdc[20036](info): commencing operation
krb5kdc: starting...
W20260811 03:02:02.447482 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:02:02 dist-test-slave-gjw7 krb5kdc[20036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:02:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:02:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:02:02.564074 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36131
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:35469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:36131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:02:02.673589 20058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:02.673849 20058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:02.673888 20058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:02.677774 20058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:02.677824 20058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:02.677841 20058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:02.677855 20058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:02.677870 20058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:02.682984 20058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:35469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:36131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:02.683956 20058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:02.684916 20058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:02:02.693089 20071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:02.693130 20065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:02.693138 20068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:02.693454 20058 server_base.cc:1034] running on GCE node
I20260811 03:02:02.693990 20058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:02.695042 20058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:02.696195 20058 hybrid_clock.cc:648] HybridClock initialized: now 1786417322696184 us; error 27 us; skew 500 ppm
Aug 11 03:02:02 dist-test-slave-gjw7 krb5kdc[20036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417322, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:02.698801 20058 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:02:02.699685 20058 webserver.cc:460] Webserver started at http://127.8.203.126:46027/ using document root <none> and password file <none>
I20260811 03:02:02.700189 20058 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:02.700246 20058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:02.700419 20058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:02.702432 20058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "8c21d17e9fd04c2795c7ac118db4d51f"
format_stamp: "Formatted at 2026-08-11 03:02:02 on dist-test-slave-gjw7"
server_key: "1f151cef24198e3f1a3daf4db2b95c5f"
server_key_iv: "aaed6957f47bcd1da3e6df9601605f56"
server_key_version: "encryptionkey@0"
I20260811 03:02:02.702940 20058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "8c21d17e9fd04c2795c7ac118db4d51f"
format_stamp: "Formatted at 2026-08-11 03:02:02 on dist-test-slave-gjw7"
server_key: "1f151cef24198e3f1a3daf4db2b95c5f"
server_key_iv: "aaed6957f47bcd1da3e6df9601605f56"
server_key_version: "encryptionkey@0"
I20260811 03:02:02.706475 20058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:02:02.708781 20080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:02.710035 20058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:02:02.710141 20058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "8c21d17e9fd04c2795c7ac118db4d51f"
format_stamp: "Formatted at 2026-08-11 03:02:02 on dist-test-slave-gjw7"
server_key: "1f151cef24198e3f1a3daf4db2b95c5f"
server_key_iv: "aaed6957f47bcd1da3e6df9601605f56"
server_key_version: "encryptionkey@0"
I20260811 03:02:02.710237 20058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:02.730263 20058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:02.733275 20058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:02.733491 20058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:02.740191 20058 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:36131
I20260811 03:02:02.740209 20190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:36131 every 8 connection(s)
I20260811 03:02:02.741124 20058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:02:02.743880 20192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:02.749439 20192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f: Bootstrap starting.
I20260811 03:02:02.750409 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20058
I20260811 03:02:02.750548 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:02:02.750862 9005 external_mini_cluster.cc:1428] Setting key 353f36c50e33a4153017856798937675
I20260811 03:02:02.752306 20192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:02.753033 20192 log.cc:824] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f: Log is configured to *not* fsync() on all Append() calls
I20260811 03:02:02.754940 20192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f: No bootstrap required, opened a new log
I20260811 03:02:02.758426 20192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36131 } }
I20260811 03:02:02.758642 20192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:02:02.758697 20192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c21d17e9fd04c2795c7ac118db4d51f, State: Initialized, Role: FOLLOWER
I20260811 03:02:02.759115 20192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36131 } }
I20260811 03:02:02.759258 20192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:02:02.759322 20192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:02:02.759426 20192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:02.759754 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:02.751963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60647 (local address 127.8.203.126:36131)
0811 03:02:02.752372 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:02.752393 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:02.752565 (+ 172us) server_negotiation.cc:407] Connection header received
0811 03:02:02.753348 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:02.753374 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:02.753809 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:02.754129 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:02.754532 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:02.755528 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:02.756424 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:02.756899 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:02.757047 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:02.757905 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:02.757911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:02.757923 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:02:02.758298 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:02.758359 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:02:02.758383 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:02.758467 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:02.758606 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 03:02:02.758935 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":102,"threads_started":1}
I20260811 03:02:02.760279 20192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36131 } }
I20260811 03:02:02.760602 20192 leader_election.cc:302] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c21d17e9fd04c2795c7ac118db4d51f; no voters:
I20260811 03:02:02.760936 20192 leader_election.cc:288] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:02:02.761021 20200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:02:02.761227 20200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [term 1 LEADER]: Becoming Leader. State: Replica: 8c21d17e9fd04c2795c7ac118db4d51f, State: Running, Role: LEADER
I20260811 03:02:02.761586 20200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36131 } }
I20260811 03:02:02.762173 20192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:02.762676 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:02:02.762789 20200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c21d17e9fd04c2795c7ac118db4d51f. Latest consensus state: current_term: 1 leader_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36131 } } }
I20260811 03:02:02.762904 20200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:02.763264 20211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:02:02.763444 20201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c21d17e9fd04c2795c7ac118db4d51f" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36131 } } }
I20260811 03:02:02.763595 20201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:02.766501 20211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:02:02.769611 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:02.764855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41485 (local address 127.8.203.126:36131)
0811 03:02:02.764952 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:02.764956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:02.765418 (+ 462us) server_negotiation.cc:407] Connection header received
0811 03:02:02.765588 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:02.765591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:02.765632 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:02.765676 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:02.766058 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:02.766750 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:02.767845 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:02.768185 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:02.768298 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:02.769142 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:02.769146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:02.769149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:02:02.769163 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:02.769192 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:02:02.769206 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:02.769283 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:02.769427 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 03:02:02.769482 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 03:02:02.771864 20211 catalog_manager.cc:1347] Generated new cluster ID: ea6a4072988e437cbc3f89323a076f58
I20260811 03:02:02.771925 20211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:02:02.797282 20211 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:02:02.798250 20211 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:02:02.803062 20211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c21d17e9fd04c2795c7ac118db4d51f: Generated new TSK 0
I20260811 03:02:02.803746 20211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:02:02.807745 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20058
2026-08-11T03:02:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:02:02 dist-test-slave-gjw7 krb5kdc[20254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:02:02 dist-test-slave-gjw7 krb5kdc[20254](info): set up 2 sockets
Aug 11 03:02:02 dist-test-slave-gjw7 krb5kdc[20254](info): commencing operation
krb5kdc: starting...
W20260811 03:02:04.023628 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:02:04 dist-test-slave-gjw7 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:02:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:02:04.139317 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36791
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:37521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:36791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:02:04.247821 20274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:04.248064 20274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:04.248102 20274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:04.251942 20274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:04.251989 20274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:04.252007 20274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:04.252025 20274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:04.252043 20274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:04.256984 20274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:37521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:36791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:36791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:04.257889 20274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:04.258792 20274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:02:04.266788 20290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:04.266798 20289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:04.266798 20297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:04.266985 20274 server_base.cc:1034] running on GCE node
I20260811 03:02:04.267549 20274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:04.268561 20274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:04.269701 20274 hybrid_clock.cc:648] HybridClock initialized: now 1786417324269676 us; error 41 us; skew 500 ppm
Aug 11 03:02:04 dist-test-slave-gjw7 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:04.276196 20274 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:02:04.277050 20274 webserver.cc:460] Webserver started at http://127.8.203.126:46161/ using document root <none> and password file <none>
I20260811 03:02:04.277616 20274 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:04.277698 20274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:04.277916 20274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:04.279939 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "60f2a17c55a74cb08d3c89fb291e817b"
format_stamp: "Formatted at 2026-08-11 03:02:04 on dist-test-slave-gjw7"
server_key: "1058675bbf62f954f18202608bdf2623"
server_key_iv: "efd92ed289dae43d00a46b0a260b88b5"
server_key_version: "encryptionkey@0"
I20260811 03:02:04.280462 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "60f2a17c55a74cb08d3c89fb291e817b"
format_stamp: "Formatted at 2026-08-11 03:02:04 on dist-test-slave-gjw7"
server_key: "1058675bbf62f954f18202608bdf2623"
server_key_iv: "efd92ed289dae43d00a46b0a260b88b5"
server_key_version: "encryptionkey@0"
I20260811 03:02:04.284088 20274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 03:02:04.286369 20309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:04.287504 20274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 03:02:04.287616 20274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "60f2a17c55a74cb08d3c89fb291e817b"
format_stamp: "Formatted at 2026-08-11 03:02:04 on dist-test-slave-gjw7"
server_key: "1058675bbf62f954f18202608bdf2623"
server_key_iv: "efd92ed289dae43d00a46b0a260b88b5"
server_key_version: "encryptionkey@0"
I20260811 03:02:04.287705 20274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:04.292038 20274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:04.294668 20274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:04.294826 20274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:04.301155 20274 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:36791
I20260811 03:02:04.301168 20407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:36791 every 8 connection(s)
I20260811 03:02:04.302007 20274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:02:04.304206 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20274
I20260811 03:02:04.304338 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:02:04.304622 9005 external_mini_cluster.cc:1428] Setting key 3a724d719548d37edba8284aa1f50c09
I20260811 03:02:04.304734 20409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:04.309721 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:04.305976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43913 (local address 127.8.203.126:36791)
0811 03:02:04.306399 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:04.306414 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:04.306448 (+ 34us) server_negotiation.cc:407] Connection header received
0811 03:02:04.307283 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:04.307299 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:04.307572 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:04.307863 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:04.308567 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:04.308572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:04.308584 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:02:04.308594 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:04.308645 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:02:04.308664 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:04.308783 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:04.308854 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 03:02:04.309001 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":135,"threads_started":1}
I20260811 03:02:04.311095 20409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b: Bootstrap starting.
I20260811 03:02:04.312309 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:02:04.313391 20409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:04.314123 20409 log.cc:824] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b: Log is configured to *not* fsync() on all Append() calls
I20260811 03:02:04.315620 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:04.314146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42563 (local address 127.8.203.126:36791)
0811 03:02:04.314257 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:04.314262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:04.314439 (+ 177us) server_negotiation.cc:407] Connection header received
0811 03:02:04.314578 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:04.314580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:04.314610 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:04.314673 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:04.315308 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:04.315312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:04.315314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 03:02:04.315326 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:04.315354 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 03:02:04.315367 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:04.315450 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:04.315486 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 03:02:04.315519 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:02:04.316200 20409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b: No bootstrap required, opened a new log
I20260811 03:02:04.319533 20409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f2a17c55a74cb08d3c89fb291e817b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36791 } }
I20260811 03:02:04.319739 20409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 03:02:04.319779 20409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f2a17c55a74cb08d3c89fb291e817b, State: Initialized, Role: FOLLOWER
I20260811 03:02:04.320124 20409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f2a17c55a74cb08d3c89fb291e817b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36791 } }
I20260811 03:02:04.320232 20409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 03:02:04.320274 20409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 03:02:04.320348 20409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:04.321126 20409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f2a17c55a74cb08d3c89fb291e817b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36791 } }
I20260811 03:02:04.321477 20409 leader_election.cc:302] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60f2a17c55a74cb08d3c89fb291e817b; no voters:
I20260811 03:02:04.321741 20409 leader_election.cc:288] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 03:02:04.321854 20432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:02:04.322053 20432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [term 1 LEADER]: Becoming Leader. State: Replica: 60f2a17c55a74cb08d3c89fb291e817b, State: Running, Role: LEADER
I20260811 03:02:04.322384 20432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f2a17c55a74cb08d3c89fb291e817b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36791 } }
I20260811 03:02:04.322840 20409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:04.323850 20434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60f2a17c55a74cb08d3c89fb291e817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f2a17c55a74cb08d3c89fb291e817b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36791 } } }
I20260811 03:02:04.323979 20434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:04.324316 20437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60f2a17c55a74cb08d3c89fb291e817b. Latest consensus state: current_term: 1 leader_uuid: "60f2a17c55a74cb08d3c89fb291e817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f2a17c55a74cb08d3c89fb291e817b" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 36791 } } }
I20260811 03:02:04.324357 20449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:02:04.324924 20437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:04.327756 20449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:02:04.332993 20449 catalog_manager.cc:1347] Generated new cluster ID: 10603c6cd6bf478887788f901ecd32ba
I20260811 03:02:04.333052 20449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:02:04.343739 20449 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:02:04.344046 20449 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:02:04.352543 20449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60f2a17c55a74cb08d3c89fb291e817b: Generated new TSK 0
I20260811 03:02:04.353253 20449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 03:02:04.358125 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20274
2026-08-11T03:02:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1558 ms)
[----------] 18 tests from AuthTokenIssuingTest (28267 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:02:04 dist-test-slave-gjw7 krb5kdc[20472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:02:04 dist-test-slave-gjw7 krb5kdc[20472](info): set up 2 sockets
Aug 11 03:02:04 dist-test-slave-gjw7 krb5kdc[20472](info): commencing operation
krb5kdc: starting...
W20260811 03:02:05.584714 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:02:05 dist-test-slave-gjw7 krb5kdc[20472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:02:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:02:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:02:05.703783 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43903
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43903,127.8.203.125:32795,127.8.203.124:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:02:05.813834 20496 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:02:05.814069 20496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:05.814093 20496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:05.814148 20496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:05.817996 20496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:05.818043 20496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:05.818063 20496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:05.818090 20496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:05.818116 20496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:05.823127 20496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43903,127.8.203.125:32795,127.8.203.124:44511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:43903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:05.824081 20496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:05.825042 20496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:02:05.833091 20496 server_base.cc:1034] running on GCE node
W20260811 03:02:05.833045 20512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:05.833033 20506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:05.833017 20509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:05.833832 20496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:05.834821 20496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:05.835984 20496 hybrid_clock.cc:648] HybridClock initialized: now 1786417325835970 us; error 24 us; skew 500 ppm
Aug 11 03:02:05 dist-test-slave-gjw7 krb5kdc[20472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:05.838538 20496 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:02:05.839365 20496 webserver.cc:460] Webserver started at http://127.8.203.126:41239/ using document root <none> and password file <none>
I20260811 03:02:05.839895 20496 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:05.839962 20496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:05.840165 20496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:05.842216 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "c429ee678f1c4f6882569200fcbb8035"
format_stamp: "Formatted at 2026-08-11 03:02:05 on dist-test-slave-gjw7"
server_key: "e01159d5aa8126a36a33593f60e2e874"
server_key_iv: "b8869266b85e949bdaeddb1d49e35dd3"
server_key_version: "encryptionkey@0"
I20260811 03:02:05.842746 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "c429ee678f1c4f6882569200fcbb8035"
format_stamp: "Formatted at 2026-08-11 03:02:05 on dist-test-slave-gjw7"
server_key: "e01159d5aa8126a36a33593f60e2e874"
server_key_iv: "b8869266b85e949bdaeddb1d49e35dd3"
server_key_version: "encryptionkey@0"
I20260811 03:02:05.846279 20496 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 03:02:05.848474 20535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:05.849646 20496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 03:02:05.849756 20496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "c429ee678f1c4f6882569200fcbb8035"
format_stamp: "Formatted at 2026-08-11 03:02:05 on dist-test-slave-gjw7"
server_key: "e01159d5aa8126a36a33593f60e2e874"
server_key_iv: "b8869266b85e949bdaeddb1d49e35dd3"
server_key_version: "encryptionkey@0"
I20260811 03:02:05.849854 20496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:05.868710 20496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:05.871452 20496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:05.871632 20496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:05.878374 20496 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:43903
I20260811 03:02:05.878652 20625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:43903 every 8 connection(s)
I20260811 03:02:05.879753 20496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:02:05.889101 20626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:05.889844 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20496
I20260811 03:02:05.889993 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:02:05.890321 9005 external_mini_cluster.cc:1428] Setting key ca3b73ff80ab0c89401973154ac8c25e
I20260811 03:02:05.893636 20626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.203.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.125@KRBTEST.COM" created.
Aug 11 03:02:05 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:05.908496 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:05.896101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.8.203.126:43903)
0811 03:02:05.896520 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:05.896539 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:05.897371 (+ 832us) server_negotiation.cc:407] Connection header received
0811 03:02:05.897850 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:05.897856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:05.898044 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:05.898213 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:05.899053 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:05.899950 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:05.900978 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:05.901450 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:05.901628 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:05.904147 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:05.904152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:05.904162 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:05.904184 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:05.905644 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:05.906209 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:05.906214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:05.906215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:05.906314 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:05.906646 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:05.906648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:05.906649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:05.906991 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:05.907088 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:05.907502 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 03:02:05.907738 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
I20260811 03:02:05.908501 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:05.896101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43903 (local address 127.0.0.1:48708)
0811 03:02:05.896645 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:05.896681 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:05.897563 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:05.898232 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:05.898237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:05.898669 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:05.898914 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:05.898928 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:05.900134 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:05.900138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:05.900871 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:05.900882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:05.901596 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:05.901599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:05.901727 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:05.902317 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:05.902334 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:05.904004 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:05.905765 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:05.905776 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:05.905787 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:05.906110 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:05.906406 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:05.906409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:05.906411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:05.906564 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:05.907097 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:05.907104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:05.907272 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 03:02:05.907462 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 03:02:05.907845 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":79,"spinlock_wait_cycles":24448,"thread_start_us":171,"threads_started":1}
I20260811 03:02:05.911430 20626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:05.912151 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:05.911926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35540)
0811 03:02:05.912029 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:05.912095 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 03:02:05.912344 20539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
I20260811 03:02:05.913154 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:05.912943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35550)
0811 03:02:05.913042 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:05.913085 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 03:02:05.913318 20626 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:32795: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab.
Entry for principal kudu/127.8.203.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab.
I20260811 03:02:05.917389 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:32795
--webserver_interface=127.8.203.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab
--principal=kudu/127.8.203.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43903,127.8.203.125:32795,127.8.203.124:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:02:05.962565 20626 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } attempt: 1
I20260811 03:02:05.963413 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:05.963114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35566)
0811 03:02:05.963263 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:05.963324 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:02:05.964128 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:05.963942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35582)
0811 03:02:05.964029 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:05.964073 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 03:02:05.964272 20626 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:32795: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
W20260811 03:02:06.027457 20640 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:02:06.027745 20640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:06.027768 20640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:06.027812 20640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:06.031962 20640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:06.032020 20640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:06.032038 20640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:06.032053 20640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:06.032070 20640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:06.032449 20626 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } attempt: 2
I20260811 03:02:06.033237 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.032957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35586)
0811 03:02:06.033104 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.033163 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 03:02:06.033926 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.033741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35592)
0811 03:02:06.033825 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.033869 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 03:02:06.034041 20626 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:32795: Network error: Client connection negotiation failed: client connection to 127.8.203.125:32795: connect: Connection refused (error 111)
I20260811 03:02:06.037453 20640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43903,127.8.203.125:32795,127.8.203.124:44511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab
--principal=kudu/127.8.203.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:32795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.203.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:06.038419 20640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:06.039368 20640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:02:06.047657 20657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:06.047739 20640 server_base.cc:1034] running on GCE node
W20260811 03:02:06.047760 20660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:06.047657 20653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:06.048470 20640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:06.049563 20640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:06.050714 20640 hybrid_clock.cc:648] HybridClock initialized: now 1786417326050694 us; error 36 us; skew 500 ppm
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:06.053462 20640 init.cc:377] Logged in from keytab as kudu/127.8.203.125@KRBTEST.COM (short username kudu)
I20260811 03:02:06.054327 20640 webserver.cc:460] Webserver started at http://127.8.203.125:44555/ using document root <none> and password file <none>
I20260811 03:02:06.054863 20640 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:06.054941 20640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:06.055146 20640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:06.057134 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data/instance:
uuid: "07484ba7de034925bb446bcbb3e727b5"
format_stamp: "Formatted at 2026-08-11 03:02:06 on dist-test-slave-gjw7"
server_key: "147b221082484bc30588c3c16763770a"
server_key_iv: "b6146ee185d74c818a7fe4b484bdfffa"
server_key_version: "encryptionkey@0"
I20260811 03:02:06.057704 20640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/wal/instance:
uuid: "07484ba7de034925bb446bcbb3e727b5"
format_stamp: "Formatted at 2026-08-11 03:02:06 on dist-test-slave-gjw7"
server_key: "147b221082484bc30588c3c16763770a"
server_key_iv: "b6146ee185d74c818a7fe4b484bdfffa"
server_key_version: "encryptionkey@0"
I20260811 03:02:06.061192 20640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 03:02:06.063463 20680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:06.064607 20640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:02:06.064726 20640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/wal
uuid: "07484ba7de034925bb446bcbb3e727b5"
format_stamp: "Formatted at 2026-08-11 03:02:06 on dist-test-slave-gjw7"
server_key: "147b221082484bc30588c3c16763770a"
server_key_iv: "b6146ee185d74c818a7fe4b484bdfffa"
server_key_version: "encryptionkey@0"
I20260811 03:02:06.064817 20640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:06.090586 20640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:06.093325 20640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:06.093494 20640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:06.100099 20640 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.125:32795
I20260811 03:02:06.100347 20779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.125:32795 every 8 connection(s)
I20260811 03:02:06.101040 20640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
I20260811 03:02:06.103303 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20640
I20260811 03:02:06.103433 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-1/wal/instance
I20260811 03:02:06.103703 9005 external_mini_cluster.cc:1428] Setting key 3e51083aa86261e92fa2e9eb4d495d20
I20260811 03:02:06.103808 20781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:06.108237 20781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.203.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.124@KRBTEST.COM" created.
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:06.122781 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.110585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.8.203.126:43903)
0811 03:02:06.110741 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.110746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.111575 (+ 829us) server_negotiation.cc:407] Connection header received
0811 03:02:06.112494 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.112499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.112563 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.112666 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.113807 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.114359 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.115514 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.115782 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.115867 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.118365 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.118374 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.118376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.118403 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.120525 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.121128 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.121132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.121134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.121236 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.121619 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.121622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.121624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.121859 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.121967 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.122471 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.122605 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 03:02:06.123457 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.110938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43903 (local address 127.0.0.1:48724)
0811 03:02:06.111407 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.111458 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.112273 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.112795 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.112802 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.113153 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.113672 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.113685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.114470 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.114473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.115406 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.115416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.115958 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.115964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.116127 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.116659 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.116675 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.118189 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.120677 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.120683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.120695 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.121019 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.121385 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.121387 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.121388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.121530 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.121964 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.121968 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.122208 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.122361 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.122610 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260811 03:02:06.124923 20781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:06.125174 20626 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } attempt: 3
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab.
Entry for principal kudu/127.8.203.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab.
I20260811 03:02:06.132320 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:44511
--webserver_interface=127.8.203.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab
--principal=kudu/127.8.203.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:43903,127.8.203.125:32795,127.8.203.124:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:02:06.134994 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.125571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35608)
0811 03:02:06.125699 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.125713 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.125812 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.126307 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.126310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.126474 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.126598 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.126607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.127577 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.127581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.128094 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.128101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.128893 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.128897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.129000 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.129544 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.129558 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.131089 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.133360 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.133364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.133366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.133604 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.133965 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.133968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.133970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.134062 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.134519 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.134522 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.134621 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.134710 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.134839 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 03:02:06.135139 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.125767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.8.203.125:32795)
0811 03:02:06.126037 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.126044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.126055 (+ 11us) server_negotiation.cc:407] Connection header received
0811 03:02:06.126084 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.126086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.126182 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.126613 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.126714 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.127478 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.128220 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.128721 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.128872 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.131220 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.131249 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.131252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.131290 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.133217 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.133706 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.133712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.133714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.133823 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.134161 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.134164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.134165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.134435 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.134554 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.134806 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.134954 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":208,"server-negotiator.queue_time_us":216,"spinlock_wait_cycles":896,"thread_start_us":102,"threads_started":1}
I20260811 03:02:06.135334 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.125432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35604)
0811 03:02:06.125547 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.125560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.125628 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.126300 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.126302 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.126465 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.126542 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.126547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.127546 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.127549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.128195 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.128204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.128898 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.128901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.128987 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.129413 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.129424 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.130666 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.133733 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.133736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.133737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.134019 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.134287 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.134290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.134292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.134384 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.134782 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.134784 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.134856 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.134960 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.135079 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 03:02:06.135510 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.125533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.8.203.125:32795)
0811 03:02:06.125946 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.125956 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.125976 (+ 20us) server_negotiation.cc:407] Connection header received
0811 03:02:06.126009 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.126013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.126183 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.126310 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.126639 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.127453 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.128293 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.128681 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.128779 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.130782 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.130791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.130804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.130853 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.133233 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.134106 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.134109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.134111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.134197 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.134473 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.134476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.134478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.134665 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.134750 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.135220 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.135315 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":325,"spinlock_wait_cycles":6016,"thread_start_us":135,"threads_started":1}
I20260811 03:02:06.137028 20781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:06.137080 20626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:06.137735 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.137479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46586)
0811 03:02:06.137592 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.137672 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 03:02:06.137763 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.137496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46598)
0811 03:02:06.137664 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.137708 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260811 03:02:06.137972 20690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
I20260811 03:02:06.138370 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.138136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46610)
0811 03:02:06.138260 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.138308 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 03:02:06.138505 20626 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:44511: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
I20260811 03:02:06.138950 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.138778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46626)
0811 03:02:06.138863 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.138905 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 03:02:06.139073 20781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:44511: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
I20260811 03:02:06.169641 20626 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } attempt: 1
I20260811 03:02:06.170464 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.170111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46634)
0811 03:02:06.170285 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.170364 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 03:02:06.171195 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.170960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46636)
0811 03:02:06.171068 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.171128 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 03:02:06.171337 20626 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:44511: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
I20260811 03:02:06.188297 20781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } attempt: 1
I20260811 03:02:06.189174 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.188890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46648)
0811 03:02:06.189032 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.189095 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 03:02:06.189851 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.189672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46664)
0811 03:02:06.189754 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.189794 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 03:02:06.189967 20781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:44511: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
W20260811 03:02:06.245812 20795 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:02:06.246070 20795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:06.246094 20795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:06.246153 20795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 03:02:06.246488 20626 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } attempt: 2
I20260811 03:02:06.247308 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.246987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46680)
0811 03:02:06.247144 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.247217 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:02:06.247943 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.247747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46682)
0811 03:02:06.247835 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.247883 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 03:02:06.248052 20626 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:44511: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
W20260811 03:02:06.250516 20795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:06.250567 20795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:06.250586 20795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:06.250615 20795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:06.250643 20795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:06.255816 20795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:43903,127.8.203.125:32795,127.8.203.124:44511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab
--principal=kudu/127.8.203.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:44511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.203.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:06.256860 20795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:06.257915 20795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:02:06.258101 20781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } attempt: 2
I20260811 03:02:06.258844 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.258579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46696)
0811 03:02:06.258717 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.258774 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 03:02:06.259485 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.259298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46704)
0811 03:02:06.259387 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.259427 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 03:02:06.259598 20781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:44511: Network error: Client connection negotiation failed: client connection to 127.8.203.124:44511: connect: Connection refused (error 111)
W20260811 03:02:06.266311 20811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:06.266305 20807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:06.266386 20805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:06.266551 20795 server_base.cc:1034] running on GCE node
I20260811 03:02:06.267143 20795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:06.268249 20795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:06.269356 20795 hybrid_clock.cc:648] HybridClock initialized: now 1786417326269329 us; error 33 us; skew 500 ppm
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:06.271986 20795 init.cc:377] Logged in from keytab as kudu/127.8.203.124@KRBTEST.COM (short username kudu)
I20260811 03:02:06.272799 20795 webserver.cc:460] Webserver started at http://127.8.203.124:42231/ using document root <none> and password file <none>
I20260811 03:02:06.273341 20795 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:06.273406 20795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:06.273579 20795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:06.275540 20795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data/instance:
uuid: "1b436c7879794672b8c8561c1a68eb86"
format_stamp: "Formatted at 2026-08-11 03:02:06 on dist-test-slave-gjw7"
server_key: "1c757facad75122544ff769d14894b8f"
server_key_iv: "5c543d2033e9f73f3462a01b8a989fcd"
server_key_version: "encryptionkey@0"
I20260811 03:02:06.276065 20795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/wal/instance:
uuid: "1b436c7879794672b8c8561c1a68eb86"
format_stamp: "Formatted at 2026-08-11 03:02:06 on dist-test-slave-gjw7"
server_key: "1c757facad75122544ff769d14894b8f"
server_key_iv: "5c543d2033e9f73f3462a01b8a989fcd"
server_key_version: "encryptionkey@0"
I20260811 03:02:06.279623 20795 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 03:02:06.281937 20820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:06.283105 20795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:02:06.283212 20795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/wal
uuid: "1b436c7879794672b8c8561c1a68eb86"
format_stamp: "Formatted at 2026-08-11 03:02:06 on dist-test-slave-gjw7"
server_key: "1c757facad75122544ff769d14894b8f"
server_key_iv: "5c543d2033e9f73f3462a01b8a989fcd"
server_key_version: "encryptionkey@0"
I20260811 03:02:06.283295 20795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:06.304731 20795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:06.307915 20795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:06.308071 20795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:06.314646 20795 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.124:44511
I20260811 03:02:06.314898 20919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.124:44511 every 8 connection(s)
I20260811 03:02:06.315570 20795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
I20260811 03:02:06.318253 20920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:06.319317 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 20795
I20260811 03:02:06.319432 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786417039021389-9005-0/minicluster-data/master-2/wal/instance
I20260811 03:02:06.319693 9005 external_mini_cluster.cc:1428] Setting key 365f5586875f380f6ed55cb73ea361a5
I20260811 03:02:06.322516 20920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:06.332901 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.324406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48728 (local address 127.8.203.126:43903)
0811 03:02:06.324551 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.324556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.324831 (+ 275us) server_negotiation.cc:407] Connection header received
0811 03:02:06.324963 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.324965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.325009 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.325074 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.325726 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.326337 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.327306 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.327518 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.327603 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.329907 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.329911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.329913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.329935 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.331278 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.331711 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.331715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.331717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.331797 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.332041 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.332043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.332044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.332192 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.332265 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.332674 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.332781 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 03:02:06.333810 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.321035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.8.203.124:44511)
0811 03:02:06.321486 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.321506 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.321547 (+ 41us) server_negotiation.cc:407] Connection header received
0811 03:02:06.322342 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.322364 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.322792 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.323147 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.323828 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.325102 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.326112 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.326671 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.326816 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.329365 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.329381 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.329394 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.329421 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.330922 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.331315 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.331321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.331327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.331473 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.331747 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.331749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.331750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.332190 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.332402 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.332541 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.332813 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":113,"threads_started":1}
I20260811 03:02:06.333810 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.324337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:43903 (local address 127.0.0.1:48728)
0811 03:02:06.324710 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.324755 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.324867 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.325135 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.325141 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.325494 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.325607 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.325613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.326444 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.326447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.327193 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.327203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.327648 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.327653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.327745 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.328263 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.328285 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.329778 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.331363 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.331371 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.331376 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.331631 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.331883 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.331886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.331888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.331968 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.332281 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.332287 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.332426 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.332582 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.332907 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":99,"thread_start_us":124,"threads_started":1}
I20260811 03:02:06.334945 20920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:06.342577 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.335441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:32795 (local address 127.0.0.1:35614)
0811 03:02:06.335577 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.335587 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.335641 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.335925 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.335928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.336083 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.336198 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.336206 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.336922 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.336925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.337492 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.337501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.337930 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.337933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.338019 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.338398 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.338411 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.339475 (+ 1064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.341122 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.341125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.341127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.341368 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.341634 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.341637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.341639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.341728 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.342104 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.342107 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.342200 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.342312 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.342425 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 03:02:06.342583 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.335562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35614 (local address 127.8.203.125:32795)
0811 03:02:06.335681 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.335686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.335701 (+ 15us) server_negotiation.cc:407] Connection header received
0811 03:02:06.335745 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.335748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.335792 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.335865 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.336305 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.336816 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.337597 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.337826 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.337920 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.339602 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.339605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.339607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.339630 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.341032 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.341464 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.341468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.341470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.341550 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.341814 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.341817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.341819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.342012 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.342088 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.342345 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.342440 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:02:06.343333 20920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:06.344167 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.336268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48740 (local address 127.8.203.126:43903)
0811 03:02:06.336384 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.336389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.336401 (+ 12us) server_negotiation.cc:407] Connection header received
0811 03:02:06.336507 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.336510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.336545 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.336601 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.337158 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.337668 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.338391 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.338601 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.338679 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.340998 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.341000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.341002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.341023 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.342431 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.342863 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.342867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.342869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.342944 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.343212 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.343215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.343217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.343412 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.343611 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.343922 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.344030 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 03:02:06.347180 20626 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } attempt: 3
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
I20260811 03:02:06.350955 20781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } attempt: 3
I20260811 03:02:06.351961 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.343725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46720)
0811 03:02:06.343820 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.343833 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.343908 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.344193 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.344195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.344349 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.344427 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.344433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.345243 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.345246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.346023 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.346035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.346511 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.346514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.346588 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.346955 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.346968 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.348268 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.350575 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.350578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.350580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.350784 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.351036 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.351039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.351040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.351121 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.351523 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.351525 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.351607 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.351697 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.351814 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1}
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417325, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
I20260811 03:02:06.352357 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.343836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.8.203.124:44511)
0811 03:02:06.343920 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.343924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.343935 (+ 11us) server_negotiation.cc:407] Connection header received
0811 03:02:06.343966 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.343968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.343999 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.344059 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.344523 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.345134 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.346129 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.346412 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.346511 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.348367 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.348370 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.348372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.348392 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.350472 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.350874 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.350877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.350879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.350952 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.351203 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.351205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.351207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.351408 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.351491 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.352132 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.352217 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":35}
I20260811 03:02:06.355541 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.345274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.8.203.125:32795)
0811 03:02:06.345421 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.345425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.345438 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:02:06.345527 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.345528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.345559 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.345614 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.346240 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.346910 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.347839 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.348108 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.348201 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.350889 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.350893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.350895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.350918 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.353717 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.354168 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.354171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.354173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.354260 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.354562 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.354566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.354567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.354768 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.354957 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.355303 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.355406 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 03:02:06.355746 20920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: Bootstrap starting.
Aug 11 03:02:06 dist-test-slave-gjw7 krb5kdc[20472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417326, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
I20260811 03:02:06.357192 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.347518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46726)
0811 03:02:06.347625 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.347638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.347706 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.348253 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.348256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.348431 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.348523 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.348531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.349688 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.349691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.350174 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.350182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.350684 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.350687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.350791 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.351205 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.351219 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.352532 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.355371 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.355374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.355376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.355836 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.356162 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.356165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.356167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.356250 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.356727 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.356731 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.356830 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.356911 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.357035 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 03:02:06.357817 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.347695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.8.203.124:44511)
0811 03:02:06.348012 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.348020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.348034 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:02:06.348065 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.348068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.348105 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.348229 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.348615 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.349591 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.350279 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.350583 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.350669 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.353405 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.353408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.353411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.353435 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.355237 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.355974 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.355978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.355979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.356053 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.356379 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.356382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.356384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.356610 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.357468 (+ 858us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.357542 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.357644 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260811 03:02:06.359313 20920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:06.359355 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:02:06.360226 20920 log.cc:824] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: Log is configured to *not* fsync() on all Append() calls
I20260811 03:02:06.360419 20626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035: Bootstrap starting.
I20260811 03:02:06.360738 20586 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c429ee678f1c4f6882569200fcbb8035"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48740
I20260811 03:02:06.362623 20920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: No bootstrap required, opened a new log
I20260811 03:02:06.363097 20626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:06.363926 20626 log.cc:824] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035: Log is configured to *not* fsync() on all Append() calls
I20260811 03:02:06.364153 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.351492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:44511 (local address 127.0.0.1:46736)
0811 03:02:06.351766 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:06.351779 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:06.351848 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:06.352749 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:06.352752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:06.352936 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:06.353019 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.353024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.353989 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.353993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.354760 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:06.354773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.355385 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.355389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:06.355476 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.355930 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:06.355945 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:06.357553 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:06.362498 (+ 4945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.362501 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.362503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.362754 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.363103 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:06.363107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:06.363108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:06.363209 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:06.363651 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:06.363654 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:06.363757 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 03:02:06.363867 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 03:02:06.363998 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260811 03:02:06.364197 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.351882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46736 (local address 127.8.203.124:44511)
0811 03:02:06.352349 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.352357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.352380 (+ 23us) server_negotiation.cc:407] Connection header received
0811 03:02:06.352578 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.352581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.352645 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.352720 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.353125 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.353861 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.354925 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.355270 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.355672 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.357684 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.357688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.357690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.357712 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.362387 (+ 4675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.362850 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.362854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.362856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.362949 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.363316 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.363320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.363322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.363537 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.363624 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.363929 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.364045 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":105,"threads_started":1}
I20260811 03:02:06.366683 20781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: Bootstrap starting.
I20260811 03:02:06.366590 20920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.367388 20920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b436c7879794672b8c8561c1a68eb86, State: Initialized, Role: FOLLOWER
I20260811 03:02:06.367695 20920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.368158 20945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.368360 20945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:06.368942 20920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:06.369154 20781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:06.369900 20781 log.cc:824] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: Log is configured to *not* fsync() on all Append() calls
I20260811 03:02:06.372046 20781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: No bootstrap required, opened a new log
I20260811 03:02:06.372700 20626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035: No bootstrap required, opened a new log
I20260811 03:02:06.372566 20795 master_runner.cc:186] Error getting master registration for 127.8.203.126:43903: OK, instance_id { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" instance_seqno: 1786417325876501 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:02:06.375856 20795 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:02:06.375737 20626 raft_consensus.cc:348] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.375975 20626 raft_consensus.cc:729] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c429ee678f1c4f6882569200fcbb8035, State: Initialized, Role: FOLLOWER
W20260811 03:02:06.375936 20968 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:02:06.376044 20968 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:02:06.376027 20781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.376250 20781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07484ba7de034925bb446bcbb3e727b5, State: Initialized, Role: FOLLOWER
I20260811 03:02:06.376307 20626 consensus_queue.cc:260] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.376569 20781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.376963 20970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.377226 20970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:06.377655 20626 sys_catalog.cc:564] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:06.377701 20586 raft_consensus.cc:482] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 03:02:06.377806 20586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:06.377871 20781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:06.379099 20586 raft_consensus.cc:504] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.380087 20586 leader_election.cc:288] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [CANDIDATE]: Term 1 election: Requested vote from peers 07484ba7de034925bb446bcbb3e727b5 (127.8.203.125:32795), 1b436c7879794672b8c8561c1a68eb86 (127.8.203.124:44511)
W20260811 03:02:06.380461 20544 tablet.cc:2369] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 03:02:06.380690 20971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.380822 20971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:06.381238 20732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c429ee678f1c4f6882569200fcbb8035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "07484ba7de034925bb446bcbb3e727b5"
I20260811 03:02:06.381506 20732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:06.382094 20867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c429ee678f1c4f6882569200fcbb8035" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1b436c7879794672b8c8561c1a68eb86"
I20260811 03:02:06.382310 20867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:06.382829 20732 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c429ee678f1c4f6882569200fcbb8035 in term 1.
I20260811 03:02:06.383220 20539 leader_election.cc:302] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07484ba7de034925bb446bcbb3e727b5, c429ee678f1c4f6882569200fcbb8035; no voters:
I20260811 03:02:06.383517 20970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:02:06.383543 20867 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c429ee678f1c4f6882569200fcbb8035 in term 1.
I20260811 03:02:06.383713 20970 raft_consensus.cc:686] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [term 1 LEADER]: Becoming Leader. State: Replica: c429ee678f1c4f6882569200fcbb8035, State: Running, Role: LEADER
I20260811 03:02:06.384037 20970 consensus_queue.cc:237] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } }
I20260811 03:02:06.385835 20970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c429ee678f1c4f6882569200fcbb8035. Latest consensus state: current_term: 1 leader_uuid: "c429ee678f1c4f6882569200fcbb8035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.385937 20970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:06.387269 20640 master_runner.cc:263] No leader master found from master 07484ba7de034925bb446bcbb3e727b5
I20260811 03:02:06.387333 20640 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:02:06.387997 20496 master_runner.cc:263] No leader master found from master c429ee678f1c4f6882569200fcbb8035
I20260811 03:02:06.388067 20496 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 03:02:06.390554 21013 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:02:06.390635 21013 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:02:06.390931 21027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:02:06.394086 21027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:02:06.399643 20867 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [term 1 FOLLOWER]: Refusing update from remote peer c429ee678f1c4f6882569200fcbb8035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:02:06.400337 21023 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:02:06.404152 20945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c429ee678f1c4f6882569200fcbb8035. Latest consensus state: current_term: 1 leader_uuid: "c429ee678f1c4f6882569200fcbb8035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.404275 20945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:06.405221 20732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [term 1 FOLLOWER]: Refusing update from remote peer c429ee678f1c4f6882569200fcbb8035: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:02:06.405831 21023 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:02:06.409174 20971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c429ee678f1c4f6882569200fcbb8035. Latest consensus state: current_term: 1 leader_uuid: "c429ee678f1c4f6882569200fcbb8035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.409313 20971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:06.414789 20970 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c429ee678f1c4f6882569200fcbb8035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.414923 20970 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:06.415258 20971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c429ee678f1c4f6882569200fcbb8035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.415292 21035 mvcc.cc:205] Tried to move back new op lower bound from 7317165368925003776 to 7317165368872521728. Current Snapshot: MvccSnapshot[applied={T|T < 7317165368925003776}]
I20260811 03:02:06.415380 20971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:06.417577 21040 mvcc.cc:205] Tried to move back new op lower bound from 7317165368925003776 to 7317165368872521728. Current Snapshot: MvccSnapshot[applied={T|T < 7317165368925003776}]
I20260811 03:02:06.418293 21023 sys_catalog.cc:455] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c429ee678f1c4f6882569200fcbb8035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.418416 21023 sys_catalog.cc:458] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:06.419539 20945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c429ee678f1c4f6882569200fcbb8035" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c429ee678f1c4f6882569200fcbb8035" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 43903 } } peers { permanent_uuid: "07484ba7de034925bb446bcbb3e727b5" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 32795 } } peers { permanent_uuid: "1b436c7879794672b8c8561c1a68eb86" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 44511 } } }
I20260811 03:02:06.419672 20945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:06.421120 21038 mvcc.cc:205] Tried to move back new op lower bound from 7317165368925003776 to 7317165368872521728. Current Snapshot: MvccSnapshot[applied={T|T < 7317165368925003776}]
I20260811 03:02:06.421473 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.389777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.8.203.124:44511)
0811 03:02:06.389900 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.389904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.397562 (+ 7658us) server_negotiation.cc:407] Connection header received
0811 03:02:06.397761 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.397765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.397815 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.397879 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.400329 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.401138 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.405699 (+ 4561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.406093 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.406172 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.413114 (+ 6942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.413118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.413122 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.413153 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.414787 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.418439 (+ 3652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.418444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.418446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.418543 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.419517 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.419521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.419523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.419792 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.419920 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.421157 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.421312 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:02:06.423187 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.390263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.8.203.125:32795)
0811 03:02:06.390367 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.390371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.399111 (+ 8740us) server_negotiation.cc:407] Connection header received
0811 03:02:06.399285 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.399288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.399335 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.399386 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.402940 (+ 3554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.403700 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.408790 (+ 5090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.409140 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.409212 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.415307 (+ 6095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.415311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.415314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.415343 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.416932 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.419122 (+ 2190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.419126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.419127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.419218 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.420467 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.420471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.420473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.420695 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.420799 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.422919 (+ 2120us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.423039 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 03:02:06.427172 21027 catalog_manager.cc:1347] Generated new cluster ID: 8e2d78a5a615496b8cd295d85d984810
I20260811 03:02:06.427232 21027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:02:06.438674 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:06.391270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.8.203.126:43903)
0811 03:02:06.391374 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.391378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.402527 (+ 11149us) server_negotiation.cc:407] Connection header received
0811 03:02:06.402691 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.402695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.402741 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.402790 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.406890 (+ 4100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.407639 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.412038 (+ 4399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:06.412394 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:06.412466 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:06.418429 (+ 5963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:06.418433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:06.418436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:06.418464 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:06.420972 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.424011 (+ 3039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.424015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.424017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.424107 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:06.437418 (+ 13311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:06.437423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:06.437425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:06.437703 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:06.437854 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:06.438339 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 03:02:06.438496 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 03:02:06.454589 21027 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:02:06.455998 21027 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:02:06.466570 21027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c429ee678f1c4f6882569200fcbb8035: Generated new TSK 0
I20260811 03:02:06.467942 21027 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:02:06.504170 20937 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.501788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.8.203.124:44511)
0811 03:02:06.501947 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.501953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.503129 (+ 1176us) server_negotiation.cc:407] Connection header received
0811 03:02:06.503317 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.503321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.503372 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.503423 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.504036 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46754: BlockingRecv error: recv got EOF from 127.0.0.1:46754 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 03:02:06.505767 20635 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.502485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.8.203.126:43903)
0811 03:02:06.502597 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.502601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.504819 (+ 2218us) server_negotiation.cc:407] Connection header received
0811 03:02:06.505062 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.505066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.505113 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.505159 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.505647 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48748: BlockingRecv error: recv got EOF from 127.0.0.1:48748 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 03:02:06.507753 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.506492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.8.203.125:32795)
0811 03:02:06.506601 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.506605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.506911 (+ 306us) server_negotiation.cc:407] Connection header received
0811 03:02:06.507075 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.507078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.507126 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.507172 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.507641 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35624: BlockingRecv error: recv got EOF from 127.0.0.1:35624 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 03:02:06.514472 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.510659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.8.203.125:32795)
0811 03:02:06.510773 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.510777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.513555 (+ 2778us) server_negotiation.cc:407] Connection header received
0811 03:02:06.513758 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.513761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.513830 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.513893 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.514376 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35630: BlockingRecv error: recv got EOF from 127.0.0.1:35630 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 03:02:06.519560 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.518151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.8.203.125:32795)
0811 03:02:06.518245 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.518249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.518683 (+ 434us) server_negotiation.cc:407] Connection header received
0811 03:02:06.518879 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.518883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.518932 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.518981 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.519466 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35632: BlockingRecv error: recv got EOF from 127.0.0.1:35632 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 03:02:06.525269 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.523866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.8.203.125:32795)
0811 03:02:06.523965 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.523969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.524412 (+ 443us) server_negotiation.cc:407] Connection header received
0811 03:02:06.524600 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.524603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.524650 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.524697 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.525170 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35642: BlockingRecv error: recv got EOF from 127.0.0.1:35642 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 03:02:06.532964 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.531498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.8.203.125:32795)
0811 03:02:06.531622 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.531625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.532110 (+ 485us) server_negotiation.cc:407] Connection header received
0811 03:02:06.532283 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.532286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.532333 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.532381 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.532872 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35654: BlockingRecv error: recv got EOF from 127.0.0.1:35654 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 03:02:06.544924 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.543356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.8.203.125:32795)
0811 03:02:06.543459 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.543463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.543964 (+ 501us) server_negotiation.cc:407] Connection header received
0811 03:02:06.544165 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.544168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.544219 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.544271 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.544810 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35658: BlockingRecv error: recv got EOF from 127.0.0.1:35658 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 03:02:06.565132 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.563593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.8.203.125:32795)
0811 03:02:06.563702 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.563707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.564206 (+ 499us) server_negotiation.cc:407] Connection header received
0811 03:02:06.564392 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.564396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.564448 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.564496 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.565018 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35668: BlockingRecv error: recv got EOF from 127.0.0.1:35668 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 03:02:06.600793 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.599675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.8.203.125:32795)
0811 03:02:06.599823 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.599828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.599927 (+ 99us) server_negotiation.cc:407] Connection header received
0811 03:02:06.600057 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.600060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.600123 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.600226 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.600644 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35682: BlockingRecv error: recv got EOF from 127.0.0.1:35682 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 03:02:06.667994 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.666929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.8.203.125:32795)
0811 03:02:06.667062 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.667065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.667199 (+ 134us) server_negotiation.cc:407] Connection header received
0811 03:02:06.667326 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.667329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.667379 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.667465 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.667891 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35696: BlockingRecv error: recv got EOF from 127.0.0.1:35696 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 03:02:06.799201 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:06.798121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.8.203.125:32795)
0811 03:02:06.798298 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:06.798302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:06.798394 (+ 92us) server_negotiation.cc:407] Connection header received
0811 03:02:06.798546 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:06.798549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:06.798602 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:06.798664 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:06.799064 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35704: BlockingRecv error: recv got EOF from 127.0.0.1:35704 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 03:02:07.058324 20794 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:07.057234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.8.203.125:32795)
0811 03:02:07.057389 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:07.057394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:07.057460 (+ 66us) server_negotiation.cc:407] Connection header received
0811 03:02:07.057617 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:07.057621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:07.057671 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:07.057736 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:07.058198 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35708: BlockingRecv error: recv got EOF from 127.0.0.1:35708 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 03:02:07.377468 20968 catalog_manager.cc:1259] Loaded cluster ID: 8e2d78a5a615496b8cd295d85d984810
I20260811 03:02:07.377550 20968 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: loading cluster ID for follower catalog manager: success
I20260811 03:02:07.379467 20968 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: acquiring CA information for follower catalog manager: success
I20260811 03:02:07.380673 20968 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b436c7879794672b8c8561c1a68eb86: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:02:07.391800 21013 catalog_manager.cc:1259] Loaded cluster ID: 8e2d78a5a615496b8cd295d85d984810
I20260811 03:02:07.391875 21013 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: loading cluster ID for follower catalog manager: success
I20260811 03:02:07.393533 21013 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: acquiring CA information for follower catalog manager: success
I20260811 03:02:07.394644 21013 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 07484ba7de034925bb446bcbb3e727b5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:02:07.575712 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:07.572238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.8.203.125:32795)
0811 03:02:07.572475 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:07.572479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:07.572498 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:02:07.572641 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:07.572643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:07.572770 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:07.572875 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:07.573080 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.573645 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.574450 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.574659 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.574728 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:07.574970 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:07.575412 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:07.575497 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:07.575537 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 03:02:07.575586 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":81,"threads_started":1}
I20260811 03:02:07.581736 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:07.578589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.8.203.124:44511)
0811 03:02:07.578809 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:07.578813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:07.578826 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:02:07.578945 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:07.578947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:07.579051 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:07.579231 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:07.579421 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.579914 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.580620 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.580819 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.580889 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:07.581074 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:07.581452 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:07.581528 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:07.581584 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 03:02:07.581627 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260811 03:02:07.587970 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:07.584434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35730 (local address 127.8.203.125:32795)
0811 03:02:07.584544 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:07.584548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:07.584683 (+ 135us) server_negotiation.cc:407] Connection header received
0811 03:02:07.584823 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:07.584826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:07.584871 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:07.584935 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:07.585148 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.585842 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.586884 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.587110 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.587198 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:07.587446 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:07.587517 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:07.587576 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:07.587696 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 03:02:07.587742 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:02:07.588150 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:07.584262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46778 (local address 127.8.203.124:44511)
0811 03:02:07.584448 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:07.584453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:07.584739 (+ 286us) server_negotiation.cc:407] Connection header received
0811 03:02:07.584860 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:07.584863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:07.584903 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:07.584963 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:07.585244 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.585830 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.586917 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.587242 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.587343 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:07.587588 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:07.587658 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:07.587716 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:07.587982 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 03:02:07.588032 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 03:02:07.589277 21193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:07.584181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.8.203.126:43903)
0811 03:02:07.584680 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:07.584684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:07.584964 (+ 280us) server_negotiation.cc:407] Connection header received
0811 03:02:07.585059 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:07.585061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:07.585194 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:07.585324 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:07.585542 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.586319 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.587604 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:07.587945 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:07.588034 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:07.588692 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:07.588957 (+ 265us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:07.589025 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:07.589096 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 03:02:07.589140 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":74,"threads_started":1}
I20260811 03:02:07.591809 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20496
I20260811 03:02:07.608278 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20640
I20260811 03:02:07.622274 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 20795
2026-08-11T03:02:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3252 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 03:02:07 dist-test-slave-gjw7 krb5kdc[21201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 03:02:07 dist-test-slave-gjw7 krb5kdc[21201](info): set up 2 sockets
Aug 11 03:02:07 dist-test-slave-gjw7 krb5kdc[21201](info): commencing operation
krb5kdc: starting...
W20260811 03:02:08.835144 9005 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/test-user.keytab.
Aug 11 03:02:08 dist-test-slave-gjw7 krb5kdc[21201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T03:02:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T03:02:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.203.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
Entry for principal kudu/127.8.203.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab.
I20260811 03:02:08.955511 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46243
--webserver_interface=127.8.203.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46243,127.8.203.125:36669,127.8.203.124:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 03:02:09.064438 21220 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:02:09.064706 21220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:09.064741 21220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:09.064801 21220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:09.068715 21220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:09.068765 21220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:09.068791 21220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:09.068813 21220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:09.068842 21220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:09.073933 21220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46243,127.8.203.125:36669,127.8.203.124:39699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.126.keytab
--principal=kudu/127.8.203.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.126:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.203.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:09.074931 21220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:09.075906 21220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:02:09.084235 21235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:09.084308 21238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:09.084417 21220 server_base.cc:1034] running on GCE node
W20260811 03:02:09.084307 21241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:09.085108 21220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:09.086189 21220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:09.087342 21220 hybrid_clock.cc:648] HybridClock initialized: now 1786417329087327 us; error 32 us; skew 500 ppm
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:09.089946 21220 init.cc:377] Logged in from keytab as kudu/127.8.203.126@KRBTEST.COM (short username kudu)
I20260811 03:02:09.090783 21220 webserver.cc:460] Webserver started at http://127.8.203.126:33363/ using document root <none> and password file <none>
I20260811 03:02:09.091300 21220 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:09.091362 21220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:09.091580 21220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:09.093595 21220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data/instance:
uuid: "5744c5b2578a4ff1886e3c406834f2f8"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "e22e19cae020f752bf087232aa6fb05d"
server_key_iv: "16971da5917548fb3623e1a8d7d05e47"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.094130 21220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/wal/instance:
uuid: "5744c5b2578a4ff1886e3c406834f2f8"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "e22e19cae020f752bf087232aa6fb05d"
server_key_iv: "16971da5917548fb3623e1a8d7d05e47"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.097743 21220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 03:02:09.099983 21253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:09.101156 21220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:02:09.101265 21220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/wal
uuid: "5744c5b2578a4ff1886e3c406834f2f8"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "e22e19cae020f752bf087232aa6fb05d"
server_key_iv: "16971da5917548fb3623e1a8d7d05e47"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.101404 21220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:09.116935 21220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:09.123693 21220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:09.123867 21220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:09.136531 21220 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.126:46243
I20260811 03:02:09.136788 21351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.126:46243 every 8 connection(s)
I20260811 03:02:09.137746 21220 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/data/info.pb
I20260811 03:02:09.140915 21352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:09.141923 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 21220
I20260811 03:02:09.142050 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-0/wal/instance
I20260811 03:02:09.142378 9005 external_mini_cluster.cc:1428] Setting key c80433e0ca0add789522581880459a77
I20260811 03:02:09.145572 21352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.203.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.125@KRBTEST.COM" created.
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:09.163756 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.147958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46243 (local address 127.0.0.1:45234)
0811 03:02:09.148541 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.148609 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.149714 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.150628 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.150634 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.151115 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.151508 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.151531 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.153151 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.153155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.154273 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.154285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.154969 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.154973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.155134 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.155893 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.155923 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.157753 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.160499 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.160508 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.160519 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.160804 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.161108 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.161111 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.161113 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.161248 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.162008 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.162014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.162160 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.162325 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.162688 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":42,"thread_start_us":123,"threads_started":1}
I20260811 03:02:09.163756 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.148927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45234 (local address 127.8.203.126:46243)
0811 03:02:09.149197 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.149207 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.149252 (+ 45us) server_negotiation.cc:407] Connection header received
0811 03:02:09.150107 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.150114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.150419 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.150610 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.151683 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.153024 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.154386 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.154819 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.154969 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.157894 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.157901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.157914 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.157936 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.160391 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.160907 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.160913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.160915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.161021 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.161372 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.161375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.161377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.161825 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.161963 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.162444 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.162759 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":191,"server-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260811 03:02:09.166539 21352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:09.167311 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.167011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60876)
0811 03:02:09.167164 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.167240 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 03:02:09.167598 21255 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
I20260811 03:02:09.168538 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.168311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60888)
0811 03:02:09.168436 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.168478 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 03:02:09.168679 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:36669: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab.
Entry for principal kudu/127.8.203.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab.
I20260811 03:02:09.173776 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:36669
--webserver_interface=127.8.203.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab
--principal=kudu/127.8.203.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46243,127.8.203.125:36669,127.8.203.124:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:02:09.217917 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } attempt: 1
I20260811 03:02:09.218844 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.218482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60896)
0811 03:02:09.218664 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.218746 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 03:02:09.219611 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.219390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60912)
0811 03:02:09.219484 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.219537 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 03:02:09.219761 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:36669: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
W20260811 03:02:09.283152 21368 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:02:09.283427 21368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:09.283450 21368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:09.283494 21368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:09.287359 21368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:09.287416 21368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:09.287433 21368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:09.287454 21368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:09.287468 21368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:09.287931 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } attempt: 2
I20260811 03:02:09.288823 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.288454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60916)
0811 03:02:09.288633 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.288708 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 03:02:09.289565 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.289354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60922)
0811 03:02:09.289450 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.289500 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 03:02:09.289690 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.125:36669: Network error: Client connection negotiation failed: client connection to 127.8.203.125:36669: connect: Connection refused (error 111)
I20260811 03:02:09.293012 21368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46243,127.8.203.125:36669,127.8.203.124:39699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.125.keytab
--principal=kudu/127.8.203.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.125:36669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.203.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:09.294049 21368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:09.295039 21368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 03:02:09.303226 21386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:09.303218 21385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:09.303256 21390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:09.303400 21368 server_base.cc:1034] running on GCE node
I20260811 03:02:09.304060 21368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:09.305214 21368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:09.306371 21368 hybrid_clock.cc:648] HybridClock initialized: now 1786417329306344 us; error 38 us; skew 500 ppm
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:09.311991 21368 init.cc:377] Logged in from keytab as kudu/127.8.203.125@KRBTEST.COM (short username kudu)
I20260811 03:02:09.312831 21368 webserver.cc:460] Webserver started at http://127.8.203.125:36479/ using document root <none> and password file <none>
I20260811 03:02:09.313390 21368 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:09.313460 21368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:09.313658 21368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:09.315639 21368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data/instance:
uuid: "04d24c35c14c44eaaf38b684604d1765"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "b420bc31340f12d9e48bf18a3c0984df"
server_key_iv: "b59194acfd463ee2704c3ee1ea3cdcd4"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.316185 21368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/wal/instance:
uuid: "04d24c35c14c44eaaf38b684604d1765"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "b420bc31340f12d9e48bf18a3c0984df"
server_key_iv: "b59194acfd463ee2704c3ee1ea3cdcd4"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.319753 21368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 03:02:09.322027 21405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:09.323143 21368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:02:09.323261 21368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/wal
uuid: "04d24c35c14c44eaaf38b684604d1765"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "b420bc31340f12d9e48bf18a3c0984df"
server_key_iv: "b59194acfd463ee2704c3ee1ea3cdcd4"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.323350 21368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:09.358753 21368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:09.361608 21368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:09.361793 21368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:09.368569 21368 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.125:36669
I20260811 03:02:09.368816 21487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.125:36669 every 8 connection(s)
I20260811 03:02:09.369561 21368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/data/info.pb
I20260811 03:02:09.370000 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 21368
I20260811 03:02:09.370123 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-1/wal/instance
I20260811 03:02:09.370460 9005 external_mini_cluster.cc:1428] Setting key 9e0a961b1e2538f3cea1dba01623aef5
I20260811 03:02:09.372954 21489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:09.376713 21489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.203.124@KRBTEST.COM; defaulting to no policy
I20260811 03:02:09.380832 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.203.124@KRBTEST.COM" created.
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
I20260811 03:02:09.397864 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.381315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60926)
0811 03:02:09.381428 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.381442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.381510 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.382195 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.382198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.382390 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.382540 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.382549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.385440 (+ 2891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.385444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.386233 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.386246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.387087 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.387091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.387201 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.387705 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.387720 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.389152 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.396114 (+ 6962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.396118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.396120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.396318 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.396664 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.396667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.396669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.396748 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.397347 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.397350 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.397450 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.397539 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.397679 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 03:02:09.398478 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.378626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.8.203.126:46243)
0811 03:02:09.378746 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.378750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.379831 (+ 1081us) server_negotiation.cc:407] Connection header received
0811 03:02:09.380740 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.380744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.380812 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.380897 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.382465 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.383199 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.384695 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.385178 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.385324 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.388456 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.388460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.388464 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.388491 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.395336 (+ 6845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.396258 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.396262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.396264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.396361 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.396788 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.396792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.396794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.397021 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.397110 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.398244 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.398364 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 03:02:09.398782 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.381418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.8.203.125:36669)
0811 03:02:09.381777 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.381786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.381805 (+ 19us) server_negotiation.cc:407] Connection header received
0811 03:02:09.381840 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.381844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.382039 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.382797 (+ 758us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.383006 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.385269 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.386394 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.386928 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.387434 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.389351 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.389360 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.389377 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.389405 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.396014 (+ 6609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.396425 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.396432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.396434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.396572 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.396831 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.396835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.396837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.397229 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.397383 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.397641 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.397856 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":173,"threads_started":1}
I20260811 03:02:09.399310 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.379170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46243 (local address 127.0.0.1:45246)
0811 03:02:09.379686 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.379728 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.380533 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.381117 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.381127 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.381659 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.382309 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.382326 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.383328 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.383331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.384511 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.384524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.385533 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.385537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.385747 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.386420 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.386442 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.388295 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.395480 (+ 7185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.395490 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.395504 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.396138 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.396499 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.396502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.396504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.396692 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.397122 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.397130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.397318 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.397500 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.399157 (+ 1657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":1319,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.203.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab.
Entry for principal kudu/127.8.203.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab.
I20260811 03:02:09.400574 21489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:09.400738 9005 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1jpVEs/build/debug/bin/kudu
/tmp/dist-test-task1jpVEs/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:39699
--webserver_interface=127.8.203.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab
--principal=kudu/127.8.203.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.203.84:38503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.203.126:46243,127.8.203.125:36669,127.8.203.124:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 03:02:09.401125 21352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:09.401818 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.401559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47874)
0811 03:02:09.401710 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.401757 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 03:02:09.402565 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.402299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47882)
0811 03:02:09.402479 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.402517 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260811 03:02:09.402714 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:39699: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
I20260811 03:02:09.408730 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.401063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60928)
0811 03:02:09.401180 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.401192 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.401318 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.401554 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.401556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.401703 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.401800 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.401806 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.402825 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.402827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.403564 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.403575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.404103 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.404107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.404197 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.404581 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.404591 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.405830 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.407506 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.407509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.407510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.407685 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.407950 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.407952 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.407953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.408015 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.408386 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.408388 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.408461 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.408530 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.408618 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:02:09.408885 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.401091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.8.203.125:36669)
0811 03:02:09.401390 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.401394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.401405 (+ 11us) server_negotiation.cc:407] Connection header received
0811 03:02:09.401432 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.401435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.401471 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.401528 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.401981 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.402713 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.403675 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.403979 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.404080 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.405962 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.405966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.405968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.405994 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.407406 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.407777 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.407781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.407783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.407866 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.408086 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.408089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.408091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.408307 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.408413 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.408602 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.408710 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260811 03:02:09.409548 21489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:09.410218 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.409933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47884)
0811 03:02:09.410043 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.410153 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 03:02:09.410732 21406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
I20260811 03:02:09.411516 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.411292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47896)
0811 03:02:09.411405 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.411457 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 03:02:09.411665 21489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:39699: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
I20260811 03:02:09.433861 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } attempt: 1
I20260811 03:02:09.434715 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.434382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47904)
0811 03:02:09.434585 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.434643 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 03:02:09.435437 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.435217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47918)
0811 03:02:09.435328 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.435384 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 03:02:09.435551 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:39699: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
I20260811 03:02:09.460913 21489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } attempt: 1
I20260811 03:02:09.461824 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.461517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47930)
0811 03:02:09.461673 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.461736 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 03:02:09.462507 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.462315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47940)
0811 03:02:09.462405 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.462448 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 03:02:09.462625 21489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:39699: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
I20260811 03:02:09.510689 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } attempt: 2
I20260811 03:02:09.511512 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.511201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47952)
0811 03:02:09.511355 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.511417 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 03:02:09.512353 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.512129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47962)
0811 03:02:09.512242 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.512286 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 03:02:09.512470 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:39699: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
W20260811 03:02:09.516233 21505 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 03:02:09.516499 21505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 03:02:09.516541 21505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 03:02:09.516603 21505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 03:02:09.520547 21505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 03:02:09.520602 21505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 03:02:09.520628 21505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 03:02:09.520663 21505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 03:02:09.520690 21505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 03:02:09.525876 21505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.203.84:38503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.203.126:46243,127.8.203.125:36669,127.8.203.124:39699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/krb5kdc/kudu_127.8.203.124.keytab
--principal=kudu/127.8.203.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.203.124:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.203.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 03:02:09.526883 21505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 03:02:09.527884 21505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 03:02:09.530753 21489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } attempt: 2
I20260811 03:02:09.531472 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.531214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47976)
0811 03:02:09.531343 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.531398 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 03:02:09.532167 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.531948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:47982)
0811 03:02:09.532053 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.532104 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 03:02:09.532344 21489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.203.124:39699: Network error: Client connection negotiation failed: client connection to 127.8.203.124:39699: connect: Connection refused (error 111)
W20260811 03:02:09.536268 21521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:09.536259 21516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 03:02:09.536240 21517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 03:02:09.536728 21505 server_base.cc:1034] running on GCE node
I20260811 03:02:09.537276 21505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 03:02:09.538358 21505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 03:02:09.539520 21505 hybrid_clock.cc:648] HybridClock initialized: now 1786417329539501 us; error 28 us; skew 500 ppm
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 03:02:09.542196 21505 init.cc:377] Logged in from keytab as kudu/127.8.203.124@KRBTEST.COM (short username kudu)
I20260811 03:02:09.543023 21505 webserver.cc:460] Webserver started at http://127.8.203.124:45363/ using document root <none> and password file <none>
I20260811 03:02:09.543540 21505 fs_manager.cc:359] Metadata directory not provided
I20260811 03:02:09.543596 21505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 03:02:09.543761 21505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 03:02:09.545748 21505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data/instance:
uuid: "2fd55bbaf80340359fb7d0093c97ca9b"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "2804f82b84443949a7667931521a0a1f"
server_key_iv: "fc71fd17fc6820d0b4f292e0e76ff8a4"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.546252 21505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/wal/instance:
uuid: "2fd55bbaf80340359fb7d0093c97ca9b"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "2804f82b84443949a7667931521a0a1f"
server_key_iv: "fc71fd17fc6820d0b4f292e0e76ff8a4"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.549702 21505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 03:02:09.551937 21534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 03:02:09.553071 21505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 03:02:09.553177 21505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data,/tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/wal
uuid: "2fd55bbaf80340359fb7d0093c97ca9b"
format_stamp: "Formatted at 2026-08-11 03:02:09 on dist-test-slave-gjw7"
server_key: "2804f82b84443949a7667931521a0a1f"
server_key_iv: "fc71fd17fc6820d0b4f292e0e76ff8a4"
server_key_version: "encryptionkey@0"
I20260811 03:02:09.553256 21505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 03:02:09.563596 21505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 03:02:09.566599 21505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 03:02:09.566743 21505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 03:02:09.573114 21505 rpc_server.cc:307] RPC server started. Bound to: 127.8.203.124:39699
I20260811 03:02:09.573390 21624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.203.124:39699 every 8 connection(s)
I20260811 03:02:09.574108 21505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/data/info.pb
I20260811 03:02:09.576814 21626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 03:02:09.578186 9005 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1jpVEs/build/debug/bin/kudu as pid 21505
I20260811 03:02:09.578320 9005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1jpVEs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786417039021389-9005-0/minicluster-data/master-2/wal/instance
I20260811 03:02:09.578647 9005 external_mini_cluster.cc:1428] Setting key 022ed201ae6e13638d4c531b78302035
I20260811 03:02:09.580996 21626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:09.591547 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.582866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.8.203.126:46243)
0811 03:02:09.583007 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.583012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.583274 (+ 262us) server_negotiation.cc:407] Connection header received
0811 03:02:09.583419 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.583422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.583470 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.583549 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.584253 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.584743 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.585718 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.586021 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.586099 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.588415 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.588418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.588420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.588444 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.589899 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.590346 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.590349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.590350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.590421 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.590658 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.590660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.590661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.590836 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.590926 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.591301 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.591429 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 03:02:09.592368 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.582758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.126:46243 (local address 127.0.0.1:45260)
0811 03:02:09.583150 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.583200 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.583321 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.583602 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.583607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.583974 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.584122 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.584130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.584866 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.584869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.585605 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.585616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.586165 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.586169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.586282 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.586777 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.586797 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.588273 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.589988 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.589996 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.590002 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.590249 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.590501 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.590504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.590506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.590586 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.590938 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.590944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.591070 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.591208 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.591486 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":93,"thread_start_us":145,"threads_started":1}
I20260811 03:02:09.592382 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.579898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.8.203.124:39699)
0811 03:02:09.580280 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.580294 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.580332 (+ 38us) server_negotiation.cc:407] Connection header received
0811 03:02:09.580967 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.580986 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.581375 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.581656 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.582538 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.583765 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.584709 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.585227 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.585395 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.587903 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.587910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.587923 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.587948 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.589455 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.589902 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.589906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.589910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.590060 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.590307 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.590310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.590312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.590748 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.590902 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.591156 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.591423 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":243,"spinlock_wait_cycles":4352,"thread_start_us":98,"threads_started":1}
I20260811 03:02:09.593523 21626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.126@KRBTEST.COM
I20260811 03:02:09.601032 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.593948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.125:36669 (local address 127.0.0.1:60940)
0811 03:02:09.594045 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.594057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.594125 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.594388 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.594391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.594536 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.594635 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.594642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.595302 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.595305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.595790 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.595797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.596221 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.596225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.596310 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.596666 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.596680 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.597932 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.599738 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.599740 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.599742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.599916 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.600163 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.600166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.600168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.600240 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.600577 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.600581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.600666 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.600770 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.600883 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 03:02:09.601068 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.594064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60940 (local address 127.8.203.125:36669)
0811 03:02:09.594193 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.594197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.594210 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:02:09.594248 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.594250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.594291 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.594355 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.594740 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.595207 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.595890 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.596125 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.596212 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.598062 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.598065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.598067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.598091 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.599450 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.600007 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.600010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.600011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.600082 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.600319 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.600322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.600323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.600487 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.600570 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.600829 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.600921 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 03:02:09.601786 21626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
I20260811 03:02:09.602147 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.594720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.8.203.126:46243)
0811 03:02:09.594814 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.594818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.594915 (+ 97us) server_negotiation.cc:407] Connection header received
0811 03:02:09.595011 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.595014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.595051 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.595113 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.595636 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.596099 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.596806 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.597017 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.597098 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.599326 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.599329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.599331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.599351 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.600572 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.600990 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.600993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.600995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.601088 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.601419 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.601422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.601423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.601575 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.601773 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.601912 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.602011 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.124@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.203.125@KRBTEST.COM
I20260811 03:02:09.609442 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.602410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.8.203.124:39699)
0811 03:02:09.602750 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.602754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.602768 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:02:09.602796 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.602798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.602845 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.602910 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.603238 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.603853 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.604610 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.604825 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.604906 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.606738 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.606741 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.606743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.606762 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.607939 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.608341 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.608344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.608345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.608421 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.608680 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.608682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.608683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.608829 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.608901 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.609148 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.609283 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":274}
I20260811 03:02:09.609444 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.602198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:48000)
0811 03:02:09.602397 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.602406 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.602466 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.602925 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.602928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.603064 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.603132 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.603138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.603945 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.603948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.604507 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.604517 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.604913 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.604916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.605013 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.605461 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.605475 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.606628 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.608035 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.608038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.608039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.608258 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.608506 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.608508 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.608509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.608599 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.608914 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.608917 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.609009 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.609117 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.609237 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":17}
I20260811 03:02:09.610767 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.603248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.8.203.125:36669)
0811 03:02:09.603362 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.603366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.603379 (+ 13us) server_negotiation.cc:407] Connection header received
0811 03:02:09.603439 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.603442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.603503 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.603575 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.604063 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.604522 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.605241 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.605589 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.605684 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.607765 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.607769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.607771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.607795 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.609219 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.609685 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.609689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.609690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.609765 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.610005 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.610008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.610010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.610189 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.610377 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.610533 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.610631 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 03:02:09.611613 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } attempt: 3
I20260811 03:02:09.612775 21626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: Bootstrap starting.
I20260811 03:02:09.613927 9005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 03:02:09.615540 21626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:09.615085 21303 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5744c5b2578a4ff1886e3c406834f2f8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45272
I20260811 03:02:09.616362 21626 log.cc:824] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: Log is configured to *not* fsync() on all Append() calls
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.126@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
I20260811 03:02:09.618561 21626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: No bootstrap required, opened a new log
I20260811 03:02:09.620195 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.611989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:48002)
0811 03:02:09.612146 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.612159 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.612226 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.612546 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.612550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.612714 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.612805 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.612812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.613613 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.613616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.614290 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.614301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.614965 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.614969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.615051 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.615460 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.615473 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.616808 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.618661 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.618664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.618666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.618910 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.619219 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.619222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.619224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.619299 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.619715 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.619718 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.619808 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.619912 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.620044 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 03:02:09.620442 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.612035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48002 (local address 127.8.203.124:39699)
0811 03:02:09.612346 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.612350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.612360 (+ 10us) server_negotiation.cc:407] Connection header received
0811 03:02:09.612400 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.612402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.612437 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.612504 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.612939 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.613513 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.614423 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.614863 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.614955 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.616951 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.616955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.616957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.616979 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.618562 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.619025 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.619029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.619030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.619106 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.619387 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.619390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.619392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.619615 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.619699 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.620174 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.620305 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249}
I20260811 03:02:09.622188 21626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.622392 21626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd55bbaf80340359fb7d0093c97ca9b, State: Initialized, Role: FOLLOWER
I20260811 03:02:09.622681 21626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.623113 21638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.623241 21638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:09.623238 21352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8: Bootstrap starting.
I20260811 03:02:09.623517 21489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } attempt: 3
I20260811 03:02:09.623888 21626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:09.626219 21352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8: Neither blocks nor log segments found. Creating new log.
W20260811 03:02:09.628448 21664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:02:09.628523 21664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 03:02:09 dist-test-slave-gjw7 krb5kdc[21201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786417329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.203.125@KRBTEST.COM for kudu/127.8.203.124@KRBTEST.COM
I20260811 03:02:09.627133 21505 master_runner.cc:186] Error getting master registration for 127.8.203.126:46243: OK, instance_id { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" instance_seqno: 1786417329130068 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:02:09.631177 21505 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 03:02:09.631994 21352 log.cc:824] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8: Log is configured to *not* fsync() on all Append() calls
I20260811 03:02:09.634218 21352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8: No bootstrap required, opened a new log
I20260811 03:02:09.636126 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.623860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.203.124:39699 (local address 127.0.0.1:48014)
0811 03:02:09.624079 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 03:02:09.624092 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 03:02:09.624163 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 03:02:09.624817 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 03:02:09.624820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 03:02:09.625001 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 03:02:09.625100 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.625107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.626473 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.626476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.627232 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 03:02:09.627243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.627856 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.627860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 03:02:09.627944 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.628396 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 03:02:09.628410 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 03:02:09.629822 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 03:02:09.632669 (+ 2847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.632672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.632674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.632893 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.633604 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 03:02:09.633607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 03:02:09.633609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 03:02:09.633726 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 03:02:09.635515 (+ 1789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 03:02:09.635519 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 03:02:09.635619 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 03:02:09.635722 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 03:02:09.635956 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260811 03:02:09.636420 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.624284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48014 (local address 127.8.203.124:39699)
0811 03:02:09.624488 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.624492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.624528 (+ 36us) server_negotiation.cc:407] Connection header received
0811 03:02:09.624563 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.624589 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.624654 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.625225 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.625505 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.626346 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.627402 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.627728 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.628148 (+ 420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.629999 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.630003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.630005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.630050 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.632561 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.633403 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.633407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.633409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.633499 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.633836 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.633840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.633842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.635337 (+ 1495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.635547 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.635974 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.636166 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 03:02:09.637394 21352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.637584 21352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5744c5b2578a4ff1886e3c406834f2f8, State: Initialized, Role: FOLLOWER
I20260811 03:02:09.637884 21352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.638316 21667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.638504 21667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:09.639107 21352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 03:02:09.639561 21489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: Bootstrap starting.
I20260811 03:02:09.641399 21303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 03:02:09.641527 21303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:09.642726 21303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.643644 21303 leader_election.cc:288] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [CANDIDATE]: Term 1 election: Requested vote from peers 04d24c35c14c44eaaf38b684604d1765 (127.8.203.125:36669), 2fd55bbaf80340359fb7d0093c97ca9b (127.8.203.124:39699)
I20260811 03:02:09.644784 21489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: Neither blocks nor log segments found. Creating new log.
I20260811 03:02:09.645673 21489 log.cc:824] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: Log is configured to *not* fsync() on all Append() calls
I20260811 03:02:09.645676 21583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5744c5b2578a4ff1886e3c406834f2f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2fd55bbaf80340359fb7d0093c97ca9b"
I20260811 03:02:09.645915 21583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:09.647214 21583 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5744c5b2578a4ff1886e3c406834f2f8 in term 1.
I20260811 03:02:09.647776 21255 leader_election.cc:302] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd55bbaf80340359fb7d0093c97ca9b, 5744c5b2578a4ff1886e3c406834f2f8; no voters:
I20260811 03:02:09.648057 21667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 03:02:09.648245 21667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [term 1 LEADER]: Becoming Leader. State: Replica: 5744c5b2578a4ff1886e3c406834f2f8, State: Running, Role: LEADER
I20260811 03:02:09.648577 21667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.650426 21667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5744c5b2578a4ff1886e3c406834f2f8. Latest consensus state: current_term: 1 leader_uuid: "5744c5b2578a4ff1886e3c406834f2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.650527 21667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:09.649399 21220 master_runner.cc:186] Error getting master registration for 127.8.203.125:36669: OK, instance_id { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" instance_seqno: 1786417329366785 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 03:02:09.652578 21220 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 03:02:09.655126 21683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:02:09.655272 21683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:02:09.655386 21701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 03:02:09.656419 21701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 03:02:09.659358 21489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: No bootstrap required, opened a new log
I20260811 03:02:09.664873 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.650522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.8.203.126:46243)
0811 03:02:09.650632 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.650637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.652602 (+ 1965us) server_negotiation.cc:407] Connection header received
0811 03:02:09.652858 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.652861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.652911 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.652962 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.654033 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.654722 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.659071 (+ 4349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.659421 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.659499 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.661391 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.661395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.661398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.661424 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.663040 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.663537 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.663541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.663543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.663629 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.663921 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.663924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.663926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.664130 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.664228 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.664595 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.664714 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 03:02:09.665422 21489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.665836 21489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d24c35c14c44eaaf38b684604d1765, State: Initialized, Role: FOLLOWER
I20260811 03:02:09.666597 21489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } }
I20260811 03:02:09.668897 21445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5744c5b2578a4ff1886e3c406834f2f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "04d24c35c14c44eaaf38b684604d1765"
I20260811 03:02:09.669122 21445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [term 0 FOLLOWER]: Advancing to term 1
I20260811 03:02:09.670354 21445 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5744c5b2578a4ff1886e3c406834f2f8 in term 1.
I20260811 03:02:09.670953 21489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 03:02:09.678025 21723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 03:02:09.678771 21723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 03:02:09.678601 21708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.678960 21708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:09.686427 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.654290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.8.203.124:39699)
0811 03:02:09.654404 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.654408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.666579 (+ 12171us) server_negotiation.cc:407] Connection header received
0811 03:02:09.667775 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.667778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.667833 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.667883 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.670643 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.671392 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.675998 (+ 4606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.676349 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.676426 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.680851 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.680855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.680858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.680887 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.682482 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.683254 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.683258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.683260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.683361 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.684154 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.684158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.684160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.684399 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.684509 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.686103 (+ 1594us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.686230 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 03:02:09.686873 21583 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [term 1 FOLLOWER]: Refusing update from remote peer 5744c5b2578a4ff1886e3c406834f2f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:02:09.687215 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:09.653657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.8.203.125:36669)
0811 03:02:09.653769 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.653773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.666365 (+ 12592us) server_negotiation.cc:407] Connection header received
0811 03:02:09.666821 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.666824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.666873 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.666923 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.668739 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.669538 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.674942 (+ 5404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:09.675281 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:09.675367 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:09.679078 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 03:02:09.679083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 03:02:09.679086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 03:02:09.679113 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 03:02:09.680690 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.683655 (+ 2965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.683659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.683660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.683742 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 03:02:09.684836 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 03:02:09.684840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 03:02:09.684842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 03:02:09.685073 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 03:02:09.685185 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:09.686963 (+ 1778us) server_negotiation.cc:299] Negotiation successful
0811 03:02:09.687084 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 03:02:09.687582 21445 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [term 1 FOLLOWER]: Refusing update from remote peer 5744c5b2578a4ff1886e3c406834f2f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 03:02:09.688635 21699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:02:09.689672 21699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 03:02:09.695545 21638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5744c5b2578a4ff1886e3c406834f2f8. Latest consensus state: current_term: 1 leader_uuid: "5744c5b2578a4ff1886e3c406834f2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.695657 21638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:09.697992 21708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5744c5b2578a4ff1886e3c406834f2f8. Latest consensus state: current_term: 1 leader_uuid: "5744c5b2578a4ff1886e3c406834f2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.698112 21708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:09.704497 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5744c5b2578a4ff1886e3c406834f2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.704630 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:09.705992 21708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5744c5b2578a4ff1886e3c406834f2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.706091 21708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:09.706773 21727 mvcc.cc:205] Tried to move back new op lower bound from 7317165382345154560 to 7317165382244323328. Current Snapshot: MvccSnapshot[applied={T|T < 7317165382345154560}]
I20260811 03:02:09.711921 21701 catalog_manager.cc:1347] Generated new cluster ID: 61a7bd9f09ea4758a05e566bb0f63dce
I20260811 03:02:09.711984 21701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 03:02:09.712208 21667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5744c5b2578a4ff1886e3c406834f2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.712306 21667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8 [sys.catalog]: This master's current role is: LEADER
I20260811 03:02:09.713999 21638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5744c5b2578a4ff1886e3c406834f2f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5744c5b2578a4ff1886e3c406834f2f8" member_type: VOTER last_known_addr { host: "127.8.203.126" port: 46243 } } peers { permanent_uuid: "04d24c35c14c44eaaf38b684604d1765" member_type: VOTER last_known_addr { host: "127.8.203.125" port: 36669 } } peers { permanent_uuid: "2fd55bbaf80340359fb7d0093c97ca9b" member_type: VOTER last_known_addr { host: "127.8.203.124" port: 39699 } } }
I20260811 03:02:09.714093 21638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 03:02:09.715111 21731 mvcc.cc:205] Tried to move back new op lower bound from 7317165382345154560 to 7317165382244323328. Current Snapshot: MvccSnapshot[applied={T|T < 7317165382345154560}]
I20260811 03:02:09.728242 21701 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 03:02:09.729470 21701 catalog_manager.cc:1504] Loading token signing keys...
I20260811 03:02:09.743253 21701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5744c5b2578a4ff1886e3c406834f2f8: Generated new TSK 0
I20260811 03:02:09.744021 21701 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 03:02:09.780922 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.777477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.8.203.125:36669)
0811 03:02:09.777626 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.777631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.779504 (+ 1873us) server_negotiation.cc:407] Connection header received
0811 03:02:09.779733 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.779737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.779794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.779855 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.780765 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55278: BlockingRecv error: recv got EOF from 127.0.0.1:55278 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 03:02:09.782138 21364 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.777889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.8.203.126:46243)
0811 03:02:09.778017 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.778022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.780164 (+ 2142us) server_negotiation.cc:407] Connection header received
0811 03:02:09.781334 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.781338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.781392 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.781446 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.782023 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48076: BlockingRecv error: recv got EOF from 127.0.0.1:48076 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 03:02:09.783777 21631 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.778568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52754 (local address 127.8.203.124:39699)
0811 03:02:09.778683 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.778687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.782630 (+ 3943us) server_negotiation.cc:407] Connection header received
0811 03:02:09.782991 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.782994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.783052 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.783108 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.783662 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52754: BlockingRecv error: recv got EOF from 127.0.0.1:52754 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 03:02:09.788444 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.786750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.8.203.125:36669)
0811 03:02:09.786851 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.786855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.787396 (+ 541us) server_negotiation.cc:407] Connection header received
0811 03:02:09.787630 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.787634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.787718 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.787776 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.788345 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55282: BlockingRecv error: recv got EOF from 127.0.0.1:55282 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 03:02:09.793515 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.791852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55292 (local address 127.8.203.125:36669)
0811 03:02:09.791962 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.791965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.792538 (+ 573us) server_negotiation.cc:407] Connection header received
0811 03:02:09.792726 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.792730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.792781 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.792831 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.793421 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55292: BlockingRecv error: recv got EOF from 127.0.0.1:55292 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 03:02:09.799129 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.797693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55294 (local address 127.8.203.125:36669)
0811 03:02:09.797787 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.797791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.798242 (+ 451us) server_negotiation.cc:407] Connection header received
0811 03:02:09.798433 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.798436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.798487 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.798538 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.799037 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55294: BlockingRecv error: recv got EOF from 127.0.0.1:55294 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 03:02:09.806619 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.805221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.8.203.125:36669)
0811 03:02:09.805357 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.805361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.805802 (+ 441us) server_negotiation.cc:407] Connection header received
0811 03:02:09.806002 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.806005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.806050 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.806096 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.806530 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55308: BlockingRecv error: recv got EOF from 127.0.0.1:55308 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 03:02:09.818140 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.816668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.8.203.125:36669)
0811 03:02:09.816761 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.816765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.817202 (+ 437us) server_negotiation.cc:407] Connection header received
0811 03:02:09.817456 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.817459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.817506 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.817551 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.818048 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55318: BlockingRecv error: recv got EOF from 127.0.0.1:55318 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 03:02:09.837522 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.836319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.8.203.125:36669)
0811 03:02:09.836434 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.836438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.836679 (+ 241us) server_negotiation.cc:407] Connection header received
0811 03:02:09.836812 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.836816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.836865 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.836933 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.837405 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55326: BlockingRecv error: recv got EOF from 127.0.0.1:55326 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 03:02:09.872864 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.871627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.8.203.125:36669)
0811 03:02:09.871753 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.871759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.871959 (+ 200us) server_negotiation.cc:407] Connection header received
0811 03:02:09.872099 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.872101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.872148 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.872230 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.872741 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55340: BlockingRecv error: recv got EOF from 127.0.0.1:55340 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 03:02:09.940074 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:09.938893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.8.203.125:36669)
0811 03:02:09.939041 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:09.939046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:09.939154 (+ 108us) server_negotiation.cc:407] Connection header received
0811 03:02:09.939317 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:09.939320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:09.939398 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:09.939472 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:09.939947 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55348: BlockingRecv error: recv got EOF from 127.0.0.1:55348 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 03:02:10.071290 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:10.070112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.8.203.125:36669)
0811 03:02:10.070250 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:10.070254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:10.070452 (+ 198us) server_negotiation.cc:407] Connection header received
0811 03:02:10.070599 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:10.070602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:10.070652 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:10.070731 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:10.071164 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55352: BlockingRecv error: recv got EOF from 127.0.0.1:55352 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 03:02:10.330523 21500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 03:02:10.329400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.8.203.125:36669)
0811 03:02:10.329546 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:10.329551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:10.329639 (+ 88us) server_negotiation.cc:407] Connection header received
0811 03:02:10.329799 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:10.329802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:10.329850 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:10.329927 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 03:02:10.330399 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55356: BlockingRecv error: recv got EOF from 127.0.0.1:55356 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 03:02:10.629837 21664 catalog_manager.cc:1259] Loaded cluster ID: 61a7bd9f09ea4758a05e566bb0f63dce
I20260811 03:02:10.629918 21664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: loading cluster ID for follower catalog manager: success
I20260811 03:02:10.631538 21664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: acquiring CA information for follower catalog manager: success
I20260811 03:02:10.632700 21664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2fd55bbaf80340359fb7d0093c97ca9b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:02:10.680130 21723 catalog_manager.cc:1259] Loaded cluster ID: 61a7bd9f09ea4758a05e566bb0f63dce
I20260811 03:02:10.680213 21723 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: loading cluster ID for follower catalog manager: success
I20260811 03:02:10.681943 21723 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: acquiring CA information for follower catalog manager: success
I20260811 03:02:10.683095 21723 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04d24c35c14c44eaaf38b684604d1765: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 03:02:10.847956 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:10.844510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.8.203.125:36669)
0811 03:02:10.844789 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:10.844793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:10.844809 (+ 16us) server_negotiation.cc:407] Connection header received
0811 03:02:10.844927 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:10.844930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:10.845062 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:10.845158 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:10.845389 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.845909 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.846690 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.846912 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.846983 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:10.847218 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:10.847652 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:10.847753 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:10.847779 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 03:02:10.847829 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260811 03:02:10.854141 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:10.850861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.8.203.124:39699)
0811 03:02:10.851090 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:10.851093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:10.851107 (+ 14us) server_negotiation.cc:407] Connection header received
0811 03:02:10.851194 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:10.851197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:10.851309 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:10.851408 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:10.851577 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.852075 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.852846 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.853056 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.853122 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:10.853368 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:10.853792 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:10.853884 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:10.853937 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 03:02:10.853985 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
I20260811 03:02:10.860638 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:10.857333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.8.203.124:39699)
0811 03:02:10.857437 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:10.857440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:10.857598 (+ 158us) server_negotiation.cc:407] Connection header received
0811 03:02:10.857695 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:10.857698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:10.857738 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:10.857796 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:10.858005 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.858587 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.859545 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.859871 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.859972 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:10.860192 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:10.860276 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:10.860338 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:10.860458 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 03:02:10.860510 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 03:02:10.861137 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:10.856908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55378 (local address 127.8.203.125:36669)
0811 03:02:10.857017 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:10.857021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:10.857582 (+ 561us) server_negotiation.cc:407] Connection header received
0811 03:02:10.857675 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:10.857678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:10.857744 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:10.857808 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:10.858144 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.858896 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.859991 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.860242 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.860315 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:10.860667 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:10.860757 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:10.860803 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:10.860966 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 03:02:10.861018 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 03:02:10.861799 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 03:02:10.856930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48084 (local address 127.8.203.126:46243)
0811 03:02:10.857150 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 03:02:10.857154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 03:02:10.857709 (+ 555us) server_negotiation.cc:407] Connection header received
0811 03:02:10.857810 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 03:02:10.857813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 03:02:10.857970 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 03:02:10.858097 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 03:02:10.858291 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.859043 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.860022 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 03:02:10.860316 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 03:02:10.860407 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 03:02:10.860800 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 03:02:10.861123 (+ 323us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 03:02:10.861202 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 03:02:10.861638 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 03:02:10.861691 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":63,"threads_started":1}
I20260811 03:02:10.864682 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 21220
I20260811 03:02:10.880296 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 21368
I20260811 03:02:10.893800 9005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1jpVEs/build/debug/bin/kudu with pid 21505
2026-08-11T03:02:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3276 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6528 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291876 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST